Pick one AI idea, not twelve. The one-job test, and why a rule beats a model when a rule will do
After the count in step one, there is a list. It is ranked by hours and money, and usually longer than expected. Add the ideas people have been saving up since they first saw a demonstration, and the list doubles. The temptation is to start all of it. My advice is to start one thing, and to check first whether that one thing needs AI at all. This is step two of five.
A typical case
Take a distributor selling to independent retailers. Say sixty orders a day arrive by email, some typed into the body of the message, most as PDFs exported from each shop's own system, every one laid out differently. Two people on the order desk key them into the ERP. The numbers are illustrative; the shape is common.
The business holds an ideas session and comes out with twelve items. A chatbot for customers. Sales forecasting. Reading emailed orders. Chasing overdue invoices. Flagging orders with a price that does not match the price list. Warning when stock will run short. Writing product descriptions. Answering "where is my order?". Spotting customers who have gone quiet. Checking freight charges. Suggesting reorder quantities. Summarising the week for the owner. Each sounds reasonable in the room.
The rule test first
Before asking which idea is best, I ask a narrower question of each one: is the input already structured, and is the decision a lookup, a threshold or a match? If it is, the answer is a validation rule in the ERP or a report, not AI. A rule is cheaper to build, it runs instantly, it gives the same answer every time, and anyone can read why it fired.
Run the list through that test. A price that does not match the price list is a comparison between two numbers the ERP already holds: a rule. Stock that will run short is on-hand stock against open orders and lead time: a report with a threshold. Overdue invoices are a date and a balance: a report and a reminder. Customers who have gone quiet are last order date against their usual frequency: a report. Freight checks against a rate table: a rule. "Where is my order?" is largely a status lookup. Reorder quantities are largely a min/max formula. Seven of twelve fall away, and that is good news. Those fixes can go in this month, and none of them needs anyone to trust a model.
The test is blunt on purpose. If a rule does the job, it is not an AI job.
The one-job test
What survives the rule test still needs a filter, because "could use AI" is not the same as "ready to try". I want five things, and I want each one written down.
A clear input. The emailed order, as it arrives, PDF or text. A clear output. A sales-order draft in the ERP, with customer, lines, quantities and requested date, waiting for a person to post it. A measured baseline. Minutes per order and errors per hundred lines, taken from the step-one count, so there is a number to beat. One owner. The order desk lead, who knows what a right order looks like and will decide whether the trial worked. A way to check each result. The draft against what was finally posted, line by line, so every miss is visible rather than averaged away.
Emailed orders pass all five. The chatbot for customers fails: there is no baseline for conversations that never happened, and no single person owns "customer questions". Forecasting fails on the check: a forecast is proved right or wrong months later, and the step-one count measured hours, not forecast error. Neither idea is bad. Neither is a first job. Product descriptions and the weekly summary wait their turn on the parked list.
What twelve at once does
Starting several at once is where I expect AI efforts to go wrong, and not because the technology fails. Nothing gets a baseline, because measuring twelve things properly takes twelve times the effort, so nobody does it. Nobody owns anything, because each idea belongs a little to everyone. And with no numbers and no owner, the choice of what to keep goes to whichever demonstration looked best on the day.
Gartner's prediction of 29 July 2024 points the same way: at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, for poor data quality, inadequate risk controls, escalating costs or unclear business value. A project with no baseline has unclear value by definition.
One job, measured and owned, gives a result that means something either way. If it works, the next idea on the list starts with a precedent and a method. If it does not, the business has lost a few weeks, not a year.
Checklist
- Every idea put through the rule test, and the rules and reports built.
- One job chosen, with its input, output, baseline, owner and check written down.
- All of it written on one page.
- The other ideas parked on a dated list, to be looked at again after the first result.
This is step two of five. Before it: Before any AI: find where the hours and the errors actually are. Next: A pilot is proven against the baseline, not in a demo. All five steps are on the AI page.