Before Public Institutions Deploy AI: Five Tests of Readiness

Artificial intelligence offers public institutions a real opportunity to improve services, analyse large volumes of information and use limited resources more effectively. It can assist in detecting anomalies, translating communications, reducing routine work and supporting better-informed decisions.

But government is not an ordinary user of technology. Its decisions may affect a citizen’s rights, livelihood, reputation, security or access to essential services. An error in a commercial recommendation may be inconvenient; an error in a public decision can have serious and lasting consequences.

After more than four decades in public service, including responsibilities connected with national security, I have learnt that a new capability should not be judged only by what it can do. We must also ask how it may fail, who may be affected and who will take responsibility.

India’s AI Governance Guidelines place trust, people, fairness, accountability, understandability and safety at the centre of responsible AI. These principles can be translated into five practical tests for public institutions.

Begin with the public purpose

The starting point should not be, “Where can we use AI?” It should be, “What public problem are we trying to solve?”

The institution must establish whether AI is actually necessary or whether a simpler solution would work equally well. It should also define the expected public benefit. Will the system reduce delays, improve accuracy, widen access or help officials make better decisions?

These objectives should be recorded before procurement begins. Without a clear purpose and a reasonable measure of success, an impressive pilot can easily become a costly system searching for a problem.

Understand the data

Every AI system depends upon data. If that data is incomplete, outdated, biased or collected without proper authority, the system may reproduce and amplify those weaknesses.

This is especially important in a country as diverse as India. Information drawn mainly from one region, language, gender, income group or social setting may not fairly represent everyone affected by the system.

Officials should understand where the data came from, whether its use is lawful, how its quality was tested and which groups may be under-represented. Privacy and security protections must be built into the system from the beginning.

Perfect data may not be possible. An honest understanding of its limitations is essential.

Keep responsibility human

Technology must not create a vacuum of responsibility.

If an AI-supported decision results in the wrongful denial of a benefit, an unfair selection or an unwarranted security concern, responsibility cannot simply be passed to the software provider. A clearly identified public authority must remain answerable.

Human oversight must also be real. It is not enough to place an official at the end of an automated process and describe it as human supervision. That official must understand the system’s limitations, have access to relevant information and possess the authority to reject its recommendation.

AI may assist human judgment. It should not become a means of avoiding it.

Preserve explanation and review

A person affected by an important public decision should ordinarily receive an understandable explanation and have access to an effective process of correction or appeal.

There will be situations—particularly in national security and law enforcement—where every operational detail cannot be disclosed. Even then, secrecy should extend only as far as genuinely necessary. Independent review, recorded reasons and safeguards against arbitrary action can still be provided.

A system that cannot be questioned may save administrative time, but it will eventually weaken public trust.

Prepare to monitor, correct and stop

Approval is not the end of scrutiny. Data patterns change, new vulnerabilities emerge and a system that works satisfactorily during a pilot may behave differently when used at scale.

Public institutions need continuing audits, security testing, incident reporting and periodic reviews of accuracy, fairness and public impact. Contracts with technology providers should clearly address data control, system updates, cybersecurity, access for audits and exit arrangements.

Most importantly, someone must have the authority to suspend the system if serious problems arise. No institution should deploy an AI system that it cannot adequately examine, correct or stop.

A question of institutional judgment

AI can make public administration more capable and responsive. Excessive caution may deny citizens useful improvements, while uncritical enthusiasm may expose them to avoidable harm. The proper course lies between these extremes.

The real measure of progress is not whether an institution is using the newest technology. It is whether that technology serves a legitimate public purpose while preserving fairness, accountability, security and human dignity.

That is not a technical decision alone. It is a test of institutional judgment.