Artificial intelligence is often sold as a tool for speed. It can sort records, summarize documents, flag unusual activity, and help a short-handed office move through routine work. For a business, a bad result may cost money or offend a customer. For a government agency, it may delay a benefit, misdirect an inspection, confuse a public record, or burden a citizen who has little choice but to deal with the system.
That difference should govern how public agencies buy and use AI.
BBC News reports that King Charles warned of the "existential danger" of AI falling into the wrong hands. The comments were made at a summit in Scotland attended by participants from Nvidia, OpenAI, and Anthropic. The warning is broad, but it points toward a practical question close to home: What controls should be in place before a public office turns an automated system loose on public business?
Write the stop rules first
Every agency considering an AI system should decide, in writing, what would cause its use to be suspended. That decision should come before a contract is signed and before staff members become dependent on the product.
A stop rule might be triggered when the system repeatedly produces factual errors, exposes information to people who should not see it, treats similar cases inconsistently, or cannot show the source behind an important conclusion. It may also be triggered when a vendor changes the product in a way that alters how public information is handled.
The point is not to predict every failure. No office can do that. The point is to prevent hesitation when something plainly goes wrong. If no one has authority to stop the system, warnings can bounce from desk to desk while the tool continues working.
The stop rule should name the official who can suspend use, the employee who must document the problem, and the person responsible for notifying affected offices. It should also say what work will continue by ordinary means while the system is offline.
Keep judgment attached to a name
Public responsibility becomes weak when an office says that a computer made the decision. A machine does not hold office, answer a letter, appear at a public meeting, or explain itself to a citizen.
For any consequential use, a named employee should remain responsible for the final action. That person should be able to see what the system produced, examine the underlying record, correct an error, and explain the result in common language. A rubber stamp is not review. If the employee lacks the time, information, or authority to disagree with the tool, the review exists only on paper.
Agencies should also separate low-stakes assistance from official judgment. Drafting a meeting summary is not the same as ranking permit applications. Sorting incoming messages is not the same as identifying suspected fraud. The greater the effect on a person's rights, duties, money, or access to public services, the stronger the human review should be.
Preserve an ordinary route for appeal
A citizen should not need technical knowledge to challenge an automated mistake. The appeal route should begin with a familiar act: a telephone call, a letter, an office visit, or a plain online form. The agency should not require the citizen to prove how the software failed before it agrees to inspect the result.
The office, not the public, chose the system. The office therefore bears the duty to maintain enough of a record to reconstruct what happened. That record should show what information went in, what recommendation came out, who reviewed it, and what final action was taken. Without that chain, an appeal can turn into guesswork.
Buy an exit, not just an entrance
Government contracts often devote great care to installation and price. AI contracts also need an exit plan. The agency should know whether its records can be exported in a useful form, how long the vendor keeps copies, what happens when service ends, and whether staff can resume the work without the product.
This is basic stewardship. A public office cannot allow a vendor's system to become the only practical way to reach, understand, or preserve the public's records.
Large warnings about AI deserve attention. But civic safety is built through smaller acts of discipline: limits written before launch, responsibility assigned to people, records kept in usable form, and a working switch that can stop the system. Public agencies do not need to reject useful tools. They do need to remain in charge of them.