AI automation: which tasks to give to software, and which not

What is the difference between RPA and AI automation?

RPA follows fixed steps: open this screen, copy that field, paste it there. That works as long as the input looks the same every time. AI automation can handle input that differs each time, such as an email in someone’s own words or an invoice in an unfamiliar format.

The trade-off: the outcome is not guaranteed to be identical every time. An AI automation therefore needs a test set that shows how often it gets things right, and a point where a person sees the doubtful cases. If your input does look the same every time, a plain rule or RPA is cheaper and more predictable.

Four levels: what the software may do alone

For each task I choose one of four levels. That choice decides more of the result than the model underneath. A task starts low and moves up a level only when the test set shows it can: the score has to reach an agreed threshold, with no serious errors during the measurement period. If the score drops later, the task goes back down a level.

0. Manual

A person does the work, the software watches and records. That is how you learn how the task really runs, exceptions included, before automating anything.

1. Draft

The software makes a draft and a person approves it before anything goes out. A draft reply to a customer question is the best-known example.

2. Within limits

The software handles the task itself within fixed limits, and a person checks samples. Whatever falls outside the limits goes to a person.

3. Autonomous

Only for steps you can undo and where a mistake does little damage. The person sets the rules, not every outcome.

How I use it myself

I run my own business on the same model. A few examples, with the level they sit at.

Assessing leads (1)

Software looks up information about a company and judges whether it fits. Nothing enters my pipeline without my approval.

LinkedIn posts (1)

An agent prepares a draft in a table. It only goes online once I have approved it; the agent cannot set that status itself.

Daily pipeline update (2)

Runs on its own every day, within fixed limits on what it may change. I check samples.

Visitor alert on this site (3)

A visit from a company network shows up as a company name in a Slack channel, without anyone approving it. A mistake costs nothing and is fixed at once. How it works is on my privacy page.

Never above 1

Pricing, client contact and the decision not to build something. Software can prepare there, but I decide.

Worked example: what does it deliver?

A fictional example, to show the sum. A wholesaler receives 400 orders a week by email. A person retypes each order into the order system, 6 minutes each on average: 40 hours a week.

After the build, the software reads the emails and prepares the orders. That takes two separate measurements. The first: how many emails does the software pass through on its own? Say 70%: then 280 go through automatically and 120 go to a person as doubtful cases, at 3 minutes each: 6 hours. The second: how many of those 280 still turn out wrong? A high score on the first measurement says nothing about that. Say 3%: then about 8 wrong orders slip in every week.

A sample check of 1 in 10 automatic orders, at 2 minutes each, takes about 1 hour, but on average finds fewer than one of those 8 errors. The customer notices the rest: a wrong delivery, a return, a phone call. At 15 minutes of repair per error, that adds 2 hours. Around 9 hours a week remain and 31 hours are freed up, with 8 customers a week who notice a mistake. If that is too many, the software has to get better or the check stricter, and that check costs time again.

From that you subtract three things: the build price, the monthly cost of the model and hosting, and the time of the owner who maintains the test set. And the 70% and the 3% are assumptions until you have measured them. Without a baseline and a test set, every number in this sum is an estimate.

What research says it delivers

The best-measured gains come from customer support. Across roughly 5,000 support agents at a large software company, an AI assistant made people 14% more productive on average, and the least experienced 35% (NBER, Generative AI at Work). The gain went mostly to those still learning the job.

When you are better off not automating

Automating costs build time, maintenance and attention. In these cases I advise against it.

Too rare

If a task comes up less than weekly, you rarely earn back the build cost.

Cannot be checked

If you cannot verify the outcome, you do not know whether the automation gets it right. Then you are building a risk.

Process is changing

If the process is about to be redesigned, automate the new process, not the old one.

The problem is elsewhere

Sometimes existing software simply is not used. Then explanation and support deliver more than new software.

Fixed input

If the input looks the same every time, a plain rule is cheaper than AI.

What the EU AI Act says

Most automation in small and mid-sized companies, such as orders, invoices or internal documents, falls into the lightest categories of the EU AI Act. Applications about people are different. In recruitment and selection, even when AI only analyses or filters applications and a recruiter decides, and in assessing the creditworthiness of individuals, Annex III of the regulation can classify the application as high risk. Human oversight is then one of the requirements, not an exemption from that classification. If software talks directly to customers, you must disclose that it is AI (Article 50).

In my levels, decisions about people stay at level 1. Whether the application counts as high risk and which requirements then apply, I check separately before anything is built.