Start with the problem, not the bot
There's a temptation to look at the latest model and ask what you can do with it. Flip that around: don't chase the model, chase the pain. The quickest path to a useful AI pilot is a clear, specific business problem with a measurable outcome. Start with the tool and you end up with a solution looking for a problem. Start with the friction and you build something people actually need.
To find the right starting point, talk to the people who do the work every day: support agents, sales reps, operations leads. Ask where they get stuck. If a task wastes hours, causes rework, or blocks revenue, it's a strong candidate for a pilot. Watch out for "pilot theatre" - projects that look good in a slide deck but are never tied to a real decision or process, and quietly fizzle because they don't change how the work gets done.
Once you've identified the pain, define success before writing a line of code. Pick one primary metric that decides if the pilot worked: time saved per task (say, cutting average triage time from 15 minutes to 5), an error rate reduction, a direct revenue impact, or a lower cost per transaction. Secondary metrics like customer satisfaction matter, but they rarely secure budget for the next phase on their own. If you can't measure it, don't pilot it.
When choosing your first pilot, look for high impact and low complexity. High expected value paired with low integration barriers gives you fast wins - email triage, content summarisation for agents, or automating manual data entry are common starting points. Make sure the pilot has an owner who can make decisions and has access to the data needed to measure outcomes.
Finally, set your stop and go criteria early. Decide what success looks like, for example a 30% reduction in handling time, and know when to pull the plug: no measurable improvement after 90 days, or the model harming a core KPI like compliance. If that happens, pause, fix the process, and try again. A short discovery phase, where you write down the problem, the metric and the owner before you build anything, saves a lot of arguing later.
Dirty data, messy results
AI rarely beats broken data. Models need internal, contextual data to produce reliable outcomes; feed a model noise and it amplifies the noise, it doesn't create value. Most pilots stall at this stage because organisations underestimate the state of their own information. Fixing data quality and structure is the unglamorous but necessary first step before any real deployment.
Be realistic about scale, too. A pilot might work on a clean, isolated dataset, but the real world is messier: your model needs to cope with the inconsistencies of day-to-day operations, not the tidy sample you tested it on.
If your data is siloed across different systems, duplicated, or full of errors, a model will struggle to learn the right patterns. Assess your data readiness before you commit to a build, so that when you do deploy, the system has a solid foundation to stand on.
To prepare, review your data sources properly: who owns the data, where it lives, and how clean it is. If you're unsure where to start, our guide on getting your data ready for AI covers the checks worth doing before you deploy.
Don't overbuild - use the right tools
You don't need to reinvent the wheel. The goal is to solve the problem, not to own the most complex infrastructure. Pick the simplest tool that solves it reliably, and only build custom when there's a real, defensible reason to.
Think of the decision as a set of traffic lights. Need results fast? Buy. An off-the-shelf SaaS tool or an LLM API gets you working in days, not months, and is usually the right route for standard use cases like customer FAQs or simple automation.
If the model or the data processing is your competitive advantage, your actual IP, then building or fine-tuning is worth considering. It's how you differentiate from competitors who are just wrapping the same standard APIs [Law.com - Build vs. Buy in the Age of Agentic AI].
Data sensitivity is another factor. Highly sensitive or regulated data often rules out hosted SaaS or unmanaged LLMs. In these cases you may need to build with strict controls, or use a private-hosted option, particularly where patient or financial data is involved.
Security matters in this decision too. LLMs and RAG systems have real attack surfaces, including prompt injection and hallucination risk, and researchers have already demonstrated self-replicating malware in AI environments [CyberMagazine - Morris II Worm Inside AI]. Understanding the specific vulnerabilities of the models you use, whether that's Claude, GPT or something else, is part of running a secure environment [DarkReading - Cybersecurity and Claude LLMs].
The most common approach we see is hybrid: off-the-shelf LLM APIs combined with your own data via RAG. It gets you the benefit of your internal documents without the cost of fine-tuning, and works well for knowledge-heavy applications. Start low-fidelity, prove the impact with off-the-shelf tools before committing to a full build, and if you do buy, require data export and interface access so you can switch vendors later. For more on this decision, see our guide on custom software development.
Design for humans, not hype
AI should make people's lives easier, not force them to learn a new interface just to get help. Focus on practical, human-centred choices that reduce friction, not product specs that only impress other engineers.
Many customers still want a human option and get frustrated when they can't reach one. Treat automation as a convenience, not a replacement, and be upfront about what the system can and can't do.
To ship something customers actually like, start simple. Automate one clear, high-value task end-to-end before expanding; complexity kills reliability. Use progressive disclosure to reveal features only when the user needs them, rather than a menu of "advanced AI" options. If the model is unsure, design a graceful fallback: a small set of safe options, or a handoff to a human.
Transparency is non-negotiable. Show when a reply is AI-generated, display confidence levels where relevant, and give an easy way to reach a human. AI works best when it supports people, functioning as an assistant rather than a full replacement; keeping a person in the loop tends to improve outcomes and build trust over time.
When a handoff to a human is needed, make it simple. Define clear escalation triggers, such as an explicit request or low model confidence, and pass along context, a short summary of the intent and previous messages, so the agent doesn't have to ask the same questions twice. Good AI UX starts with understanding what users actually need, not building clever technology nobody uses [Nielsen Norman Group - UX and AI Adoption].
This matters most for independent practices and SMEs, where the "front office" work often defines the business. If you want help thinking through where AI fits in your operation, our AI solutions overview is a good place to start.
Lock it down: privacy, security, and scam risks
You don't need a security degree to keep your business safe. You need to do the practical things that keep customer data secure and avoid a messy breach.
Start with the quick wins. Turn on multi-factor authentication everywhere: email, admin dashboards, and cloud apps. MFA stops the vast majority of account takeovers. Back up your critical data using a 3-2-1 approach: three copies, on two different media, with one copy offsite. Backups are your best defence against ransomware; a clean restore point turns a disaster into a disruption [CSO Online - How to Create a Ransomware Playbook].
Attackers are shifting away from traditional perimeter breaches toward identity theft and AI-driven social engineering, which means identity security and staff training matter more than ever [SecurityWeek - Five Cybersecurity Predictions for 2026].
This month, build a data inventory. Know where your customer data lives, whether that's a CRM, spreadsheets, or Google Drive; if you can't list it, you can't protect it. Apply the principle of least privilege by giving staff only the access they need.
Watch for OAuth and device-code phishing, where users are tricked into approving app access. Train staff never to paste codes or approve permissions they don't recognise [BleepingComputer - Microsoft 365 Accounts Targeted].
Keep it honest on the legal side too. Publish a clear privacy notice and follow it. If you handle personal or financial data, know your breach notification duties; recent breaches show how costly a slow response can be [TechCrunch - The Worst Data Breaches of 2025].
Measure, iterate, and scale without breaking things
Move forward with small, measurable experiments. Start with a single, time-boxed hypothesis: decide what you'll change, the metric you'll move, and how you'll know it worked. Keep experiments small so failures are cheap and learnings are fast. We'd rather see a business pilot two or three tools on a time-boxed basis before committing to one vendor [AFS Law - Managing AI Use in Your Organization].
Instrument everything that matters: model health, accuracy, latency, and other infrastructure metrics, plus business-cost telemetry like tokens, API calls, and time saved. That's what lets you connect technical changes to ROI later.
Watch for data drift as you run these models. The input distribution can change, or the relationship between labels can shift, and these issues erode performance quietly. Set automated checks and baselines so drift triggers an investigation rather than a surprise outage [CSO Online - Demystifying Risk in AI].
When you're ready to roll out, use safe patterns: deploy to a small percentage of users first, compare against your baseline, and ramp up only if the metrics hold. Feature flags are useful here, letting you toggle models or behaviour without redeploying code.
Keep a human in the loop. If outputs carry business risk, human review should be the default until confidence is proven, and deterministic rules should stay in place for safety-critical decisions. "AI suggests, humans decide" is a sound default while you build trust [Dark Reading - Cybersecurity Playbook for AI Adoption].
For a deeper framework on mapping these risks and controls, NIST's guidelines are a solid resource for scaling responsibly [NIST - Guidelines to Rethink Cybersecurity]. If you're building a system to support this kind of reporting, our piece on automation governance covers how to keep things sane as you scale.
Sources
- [AFS Law - Managing AI Use in Your Organization]
- [BleepingComputer - Microsoft 365 Accounts Targeted]
- [CSO Online - Demystifying Risk in AI]
- [CSO Online - How to Create a Ransomware Playbook]
- [CyberMagazine - Morris II Worm Inside AI]
- [DarkReading - Cybersecurity and Claude LLMs]
- [Dark Reading - Cybersecurity Playbook for AI Adoption]
- [Law.com - Build vs. Buy in the Age of Agentic AI]
- [Nielsen Norman Group - UX and AI Adoption]
- [NIST - Guidelines to Rethink Cybersecurity]
- [SecurityWeek - Five Cybersecurity Predictions for 2026]
- [TechCrunch - The Worst Data Breaches of 2025]