What it costs to run an AI system after launch

I have written elsewhere about what a build costs. This is the other invoice, the one that keeps coming, and it decides whether the system is still switched on in two years.

Every time your system answers a question, you pay for it. That sentence is the whole problem. Ordinary software costs a fixed amount whether anyone uses it or not. This kind costs more the more it is used, and the point of building it was to get people using it.

What you are actually paying for

The biggest part is usually the answering itself. You are charged by the amount of text involved, going in and coming back. A one line question against a short policy is cheap. The same question against a forty page contract, where the system has to read the contract before it can answer, costs many times more. People forecast how many questions there will be and forget that the size of each one is a multiplier sitting on top.

Then the parts that cost money whether or not anybody asks anything. The machine it sits on. The searchable copy of your documents, which has to be stored and rebuilt every time the documents change. The record of every question and answer you keep for audit. If the work needs the specialist chips this kind of processing runs on, you rent those by the hour and they do not stop when your staff go home.

And the people. Somebody checks the answers are still right and deals with it when they are not. In the first year that is a real monthly cost and nobody's named job, which is how it usually starts.

Usage climbs for reasons your business case never mentioned

If the thing is good, people use it more. That is success and it is also the bill going up. The rest of the growth comes from places nobody wrote down.

• People ask again. The first answer was nearly right, so they rephrase and ask twice more. Every attempt is charged.
• Another department connects to it, and your volume doubles with no new users at all.
• Somebody puts it behind a button that fires automatically on every record instead of when a person asks.
• A test is left running over a long weekend, or two systems end up calling each other in a loop, and nobody notices until the following week.

Forecasting it before you commit

You can get close in a day if there is anything running to measure. Take fifty real questions of the kind and length people will genuinely ask, measure what each one costs to answer, and take the average. Multiply by the questions a day you expect once everybody who is meant to be using it is using it. Then add for the asking twice, which in my experience runs between a fifth and a half on top.

Double that and ask whether the business case still works. If it only works at the first number, it does not work. The point is finding out whether you are looking at a few hundred dirhams a month or a figure that needs its own budget line, because those are different conversations and organisations commit to the second while picturing the first.

Write down the monthly figure at which you would switch it off, while you are at it. Nobody makes that decision well in the moment.

Put the ceiling in before it goes live

This is ordinary engineering work and it takes days rather than weeks, but only if it is asked for before launch. Afterwards it competes with everything else and slides.

A hard monthly limit that stops the system rather than emailing somebody a warning. Limits per person and per team, so one enthusiast cannot spend the quarter's budget in an afternoon. A separate login for each team that connects, so the bill can be broken down by who caused it. And repeated questions answered from a stored copy instead of asked again, because a surprising share of them are the same question.

If a supplier proposes going live without these, they have not run one of these through a busy quarter.

The signs it is getting away from you

The clearest is the bill rising faster than the number of users, which means either the average question is getting more expensive or people are asking everything twice. A weekend that costs as much as a weekday means something is running by itself. One team appearing as most of the spend usually means an automatic connection nobody is watching.

The worst sign is a month going by with nobody able to say which team spent what. The limits were never put in, whatever the plan said.

If you have a system live and cannot say what it cost per person last month, fix that first. Send me the invoice and a rough note of how many people use it, and I will tell you which of these it is.