Outgrace, home

Where FDA guidance on clinical AI software still needs a primary source

Our source set cannot say whether clinical AI software is a regulated device. Here is what the sources do cover, what they leave open, and what to ask before deciding.

The first question a clinical AI product raises is whether a regulator treats it as a medical device at all. Our source set cannot answer that. None of the publications we hold comes from the US Food and Drug Administration, and none of them says whether clinical AI software is a regulated device, what evidence the FDA expects, or what it expects when the software changes after launch. This piece therefore explains what the available sources do establish, and where a reader must go next.

How it works

NIST's AI Risk Management Framework 1.0 was released on January 26, 2023, after a consensus-driven, open process that included a request for information, several public drafts and multiple workshops, according to NIST. The framework is intended for voluntary use. It aims to build trustworthiness into the design, development, use and evaluation of AI products, services and systems. It is a method for managing risk. It is not a rulebook a device regulator enforces, and NIST's own description does not present it as one.

The White House's American AI Initiative, set out in an Executive Order, asks federal agencies to issue AI guidance across technology and industrial sectors. It says the guidance should help federal regulatory agencies develop approaches for safe and trustworthy AI adoption. It also directs NIST to lead development of technical standards for reliable, trustworthy, secure, portable and interoperable AI systems. These are instructions to agencies and to NIST. They tell a reader that regulators are expected to act. They do not say what any regulator will require of a particular product.

The World Health Organization's guidance on the ethics and governance of AI for health was produced over eighteen months by experts in ethics, digital technology, law, human rights and Ministries of Health. It names the promise of AI in health as improved diagnosis and treatment, along with health research, drug development and public health functions such as surveillance and outbreak response. It sets out six consensus principles and says ethics and human rights must sit at the heart of design, deployment and use. This is global guidance on values and governance. It does not replace a national licensing decision.

The White House document also lists diagnosing diseases among the demonstrated potential of AI, alongside transportation, education and security. Naming diagnosis as a potential use is not the same as saying a diagnostic product needs approval. A reader should keep the two apart when reading any of these documents, and should not treat a statement of potential as a statement of regulatory status.

Who it affects

According to our sources, the groups most directly addressed are federal regulatory agencies (the White House document), critical infrastructure operators (NIST's concept note on a profile for critical infrastructure, released April 7, 2026), health workers and the communities affected by health AI, and Ministries of Health (both named in the WHO guidance). The WHO recommendations call for holding public and private stakeholders accountable to health workers and affected communities. Our sources name no device makers or clinical AI companies. For a team building or buying such software, the sources describe the direction of travel and the expectations placed on others. They do not describe the rules that apply to the product itself.

What leaders get wrong

  • Treating a voluntary framework as a licence. A governance framework can be useful and still say nothing about whether a regulator has cleared a product.
  • Assuming that calling software AI settles its classification. In our judgement, whether clinical software is a device turns on what the regulator's own text says about its function, and our sources do not contain that text.
  • Reading the White House directive as a rule. It asks agencies to issue guidance. Until an agency publishes its guidance, the directive tells a team nothing about what will be required.
  • Using the WHO principles as a substitute for regulatory evidence. Ethical principles and the evidence a regulator expects are different tests, and a product can satisfy one without the other.

What to do about it

Before any classification or evidence decision, obtain the FDA's published guidance on AI-enabled device software functions and read it in full, with particular attention to the definitions and to any section on changes after launch. Our source set does not contain that document, so nothing in this piece should be used as a substitute for it. Then put three written questions to your regulatory lead and to outside regulatory counsel: whether the product's intended use makes it a device under the guidance; what evidence the guidance says that use requires; and which changes after launch need a fresh review under the guidance's change-control expectations. Record each answer in the board pack with the document and section it relies on. Where the guidance is silent or unclear, say so in writing and ask counsel to resolve it. Do not fill the gap from memory of how other markets treat similar software. Use the NIST and WHO material for governance around the product, including accountability and risk management, and keep it separate from the regulatory argument.

Newsletter

The Briefing

One email a week: the leaders we profiled, the ideas worth stealing, and nothing else.

Weekly, free, one click to leave. Read our privacy policy.