Most AI Failures Are Management Failures

Most AI failures are not model failures. They are management failures. I have come to that view from a particular vantage point. As an ISO/IEC 42001 Lead Auditor, I work inside organisations’ AI management systems: their policies, their risk registers, their impact assessments, their audit trails. That seat teaches you something the frameworks don’t. When an AI system causes harm, the model is usually the visible part of the failure. Underneath it there is almost always a test nobody ran, a decision nobody owned or a person with no way to object.

Three failures, one pattern

Consider three of the best-documented cases.

At Amazon, an experimental recruitment model learned to downgrade CVs that contained the word “women’s”. It had been trained on ten years of CVs submitted to the company, most of them from men. The model did what its data taught it. What was missing was a check, before anyone built on that data, of what it would teach.

In the Netherlands, the tax authority’s risk model for childcare benefits used nationality as one of its indicators, and tens of thousands of parents were treated as fraudsters. The model was only part of it. Families were given no clear reason for being flagged and had no effective way to appeal. A bad score became a ruined livelihood because nothing in the process could explain a decision or reverse it.

Air Canada’s website chatbot told a customer he could claim a bereavement fare after travelling. The airline’s policy said otherwise, and when the case reached a tribunal, Air Canada argued that it could not be held liable for information provided by its agents or representatives, including a chatbot. It lost. The chatbot’s error was the visible part. The failure underneath was that nobody had treated what it told customers as the airline’s own word.

None of these needed new science to prevent. Each needed an ordinary control, applied before the harm: testing by affected group, a route to explanation and redress, ownership of what the organisation’s AI says. The next article in this series, Every AI Failure Had a Missing Control, takes six such cases and the control each one lacked; the full report, When Algorithms Err, sets out eighteen.

Why governance is hard to practise

We have no shortage of material on AI governance. There are frameworks, principles, standards, regulations, guidelines, ethical codes, assessment models and an ever-growing collection of reports telling organisations what they should be doing. So why does AI governance still feel so difficult to put into practice?

Perhaps because much of the discussion falls into one of two camps: too abstract to be useful, or too legalistic to be readable. I think we need something in between: writing about what AI governance means when an organisation has to implement it, operate it, monitor it and ultimately be accountable for it. That is what this series is for. My approach will be practical, evidence-based and, where necessary, deliberately uncomfortable.

I want to look beyond the reassuring language of responsible, trustworthy and ethical AI and ask some fairly simple questions:

• What does an organisation actually have to do?
• How do you know whether the governance arrangements are working?
• What does a standard or certification actually prove?
• What happens when employees are already using AI systems the organisation doesn’t know about?
• How do you govern an AI system you didn’t build and cannot inspect?
• Who is accountable when an AI-supported decision causes harm?

And perhaps the most important: are we creating governance that genuinely reduces risk, or simply creating more policies and compliance documentation?

Frameworks are useful. They are not magic.

ISO/IEC 42001, the international standard for AI management systems, is the standard I audit against. It gives organisations that develop, provide or use AI a structured, management-system approach to governing it. That is potentially very valuable.

But a management system is not a guarantee that every AI system an organisation operates is safe, fair, accurate or ethical. Certification can provide valuable assurance that an organisation has established and operates a management system against specified requirements. It should not be confused with a certificate of “responsible AI” or “safe AI” in some absolute sense.

These distinctions matter, and you will not often hear an auditor make them in public, which is precisely why I intend to. The series will look honestly at where frameworks are strong, where they are weak, and where organisations may be tempted to treat compliance as a substitute for governance.

Regulation is only part of the story

The EU AI Act will feature prominently, not simply because it is an important piece of legislation, but because it represents a fundamentally different way of thinking about AI risk and organisational responsibility. And the question “We’re not in the EU, so why should we care?” deserves a much better answer than saying the Act has extraterritorial reach. In this series it will get one: “We’re Not in the EU” Is Not a Compliance Strategy.

AI regulation is becoming part of the global operating environment. Organisations increasingly need to understand not only which laws apply to them, but what customers, suppliers, regulators, partners and citizens will expect of their AI governance, because contracts are already carrying obligations that legislation has not yet reached.

The UK, where I am based, shows that the absence of an AI law is not the absence of AI obligations. At the time of writing it has no AI-specific statute, and the government has not introduced an AI bill. Yet since February 2026, under the Data (Use and Access) Act, organisations taking significant decisions about people solely by automated means have had to tell them, let them challenge those decisions and give them access to human intervention. Those are governance requirements.

And then there is MENA

The region will be a particular focus of the series. It is moving rapidly from AI ambition and national strategies towards large-scale adoption, with the UAE, Saudi Arabia and others investing heavily in AI capability and infrastructure. The governance signals are getting harder to miss. Dubai’s DIFC has regulated the processing of personal data by autonomous systems since 2023, and in 2024 Saudi Arabia’s data and AI authority, SDAIA, was among the first organisations anywhere to have its own AI management system certified to ISO/IEC 42001.

The governance question therefore becomes unavoidable: how do we move from AI principles and national frameworks to practical organisational governance, assurance and accountability?

For MENA organisations, regulation is not a distant prospect. It is arriving sector by sector and jurisdiction by jurisdiction. The real issue is whether they will have the governance capability, processes, skills and evidence needed when it reaches them. Organisations building that capability now, voluntarily, will meet the binding law of the coming years already standing where it points. Those that wait will be retrofitting governance under deadline pressure, which, in my experience, produces neither governance nor compliance.

One principle runs through the series

AI governance should not become another exercise in producing impressive documentation that nobody uses.

Good governance should help an organisation make better decisions about what AI to use, where to use it, how to use it, what risks to accept, what controls to implement, who is accountable, and how to demonstrate that those controls actually work. That is a much harder challenge than writing an AI policy. But it is the challenge we should be addressing, and the one this series takes on: what happens when the frameworks meet the real world.

I welcome disagreement, challenge and alternative perspectives. AI governance is evolving rapidly, and I certainly don’t claim to have all the answers. But we need more discussion about what works in practice, and what doesn’t.

Less theory. Less compliance theatre. More practical AI governance.

This is the introduction to the AI Governance in Practice series. The three cases are drawn from the Enterprise Intelligence Center report When Algorithms Err (Version 2.0, October 2026), which lists full sources for each.