1. Home
  3. AI
  5. Databricks
  7. 95% of AI Agents Never Reach Production. I Read Every Failure Study to Find Out Why.

95% of AI Agents Never Reach Production. I Read Every Failure Study to Find Out Why.

0

I sat in a backlog review a few weeks ago where a team walked us through one of the AI agents they had built. It was genuinely good. Pulled from three systems, reasoned through a messy approval flow, spat out the right answer every time in the AI Agent demo. Everyone nodded. Somebody asked when it was going to production.

Long pause. And then the honest answer, which was: we do not actually know if we can.

I have seen that pause a lot lately. I will see a hundred versions of it at the Databricks Summit this week, except there nobody says the quiet part out loud. They just show the demo and move on. So I went and read the research instead, expecting a tangle of different reasons things fall apart. That is not what I found. The studies are weirdly, almost boringly, in agreement.

The number everyone keeps quoting (and misreading)

You have probably seen the headline by now. MIT’s Project NANDA ran a study across 300-odd deployments (The GenAI Divide: State of AI in Business 2025) and found that around 5% of generative AI pilots have any measurable impact on the P&L. The rest stall. Fortune ran with it, LinkedIn lost its mind for a week, and then everyone went back to building agents anyway.

Here is the part people skip. That 95% is not failing because the models are bad. NANDA’s lead author was blunt about it: generic tools work well for one person but fall over in an enterprise because they do not integrate with the actual workflow or data. The failure is integration and foundation, not intelligence.

And it is not just MIT saying it. RAND looked at the same problem from a different angle and put the failure rate around 80%. Roughly a third of projects are abandoned before they ship, a third that ship and deliver nothing, and the rest that technically work but cost more than they are worth. Gartner went further and named the cause directly: 60% of AI projects that lack AI-ready data will be scrapped by the end of 2026, and they reckon we are already sitting at 42%.

Three different research shops. Three different methods. Same conclusion. When that happens, you should probably listen.

What actually breaks

So if the model is not the problem, what is?

It is the stuff nobody puts in a keynote. Data that nobody governs. Lineage nobody can trace. An “evaluation process” that is really just a senior engineer squinting at the output and going “yeah, looks right.” NANDA found roughly two-thirds of the failed projects underinvested in exactly this layer. The foundation. The plumbing. The unsexy part.

And here is the thing I keep coming back to, because I do not think we have fully absorbed it:

We got away with bad data for twenty years because the output was a dashboard.

Think about it. A number on a report is wrong; a human looks at it, frowns, and catches it before it does any damage. The whole BI era was built on the assumption that a person sits between the data and the decision. Messy data was survivable because we were the safety net.

An agent removes the net. You give it bad data, and it does not hand you a wrong number to sanity-check. It goes and does something. Autonomously. Then it passes its output to the next agent, which trusts it completely, and now you have a wrong decision propagating through a chain with nobody watching. The blast radius chart is no longer misleading. It is an automated mistake, repeated at machine speed.

That is why the “data readiness” stat is the whole ballgame and not a footnote. Agents took the one corner we had been cutting for two decades and made it load-bearing.

The part that stings if you work in pharma or banking

I spend most of my time in life sciences and financial services, so let me say the uncomfortable thing.

The industries with the most to gain from agents are the ones most likely to faceplant. NANDA found that buying from a specialist vendor or building on a real partnership works about 67% of the time, while rolling your own in-house succeeds maybe a third as often. Now, which industries instinctively build everything internally, behind their own walls, because compliance and risk teams will not have it any other way? Pharma. Banks. Healthcare. The exact shops writing the biggest checks are, by reflex, reaching for the approach that fails the most.

I have watched it happen. An agent that handles override processing, or reads clinical data, runs flawlessly in a notebook, and then dies the day it meets production, because the lineage was never traced and the access controls were never governed and nobody could answer the auditor’s very first question: where did this field come from, and who was allowed to see it? Not a model failure. Never was. We asked the agent to stand on a floor we had not built yet.

In a regulated shop, “it worked in the demo” and “we are cleared to ship” are separated by a canyon, and the name of that canyon is governance.

What the 5% are doing that the rest of us are not

I do not want to leave you in the graveyard, because the upside is real. The projects that do make it post a median ROI somewhere around 188%, per the RAND-based analysis. The winners are not lucky. From everything I have read and watched, they do three unglamorous things.

They decide what success means before they build. Sounds obvious. NANDA found 73% of failed projects never agreed on the metric in the first place. The winners write down the actual business outcome (the dollars, the cycle time, the headcount freed) and build backward from it.

They put the agent on governed ground. This is where the platform stops being a religious debate and starts being a source of leverage. One governance layer over your data, your models, and your agents. Lineage you can pull up in a click. (For us, this is where Unity Catalog earns its keep: when the auditor asks the field-provenance question, the answer is a lookup, not a three-week investigation.)

And they treat evaluation like engineering, not a vibe. An automated eval suite that runs every time, with real test cases, instead of the squint-and-ship method. Agent Bricks and the synthetic-eval tooling are pointed at exactly this gap. It is not flashy. It is just the difference between a pilot and a product.

None of that is a breakthrough in modeling. All of it is foundation work. Which, I think, is the entire point everyone is missing as they queue up for the next demo.

Before you build another agent, score yourself

Quick gut check. One point each, and be honest, nobody is watching:

  1. Can you trace every field your agent reads back to a governed, lineage-tracked source?
  2. Did you write down the actual P&L or operational outcome before anyone wrote code?
  3. Do you have an automated eval suite, or a senior engineer squinting?
  4. Can you answer “who is allowed to see this data” with a policy, not someone’s memory?
  5. If the agent takes the wrong action, do you find out within minutes or next quarter?
  6. Are you buying and partnering where it makes sense, or building everything in-house out of habit?
  7. Does a single governance model actually cover your data, models, and agents?

5 to 7: You are in the 5%. Go ship it. 3 to 4: You have a pilot, not a product. Close the gaps before you scale. 0 to 2: The agent is not your problem. Start with the floor it is standing on.

What this means for your next agent?

The agents on stage at the Summit are real. Nobody is faking the demos. But the gap between that stage and your production environment is real too, and it is a foundation gap, not a model gap. Everybody wants to argue about which model, which framework, which orchestration layer. The teams quietly shipping did the boring data work first, and then the agent more or less took care of itself.

That is the whole secret. I went looking for a second one, but there isn’t one.

If you are at the Summit this week, come argue with me about it. And if you run this checklist and score yourself a 2, tell me which line you are weakest on. I read every reply.

References

The figures above link to their sources inline. Full list, in case you want them in one place:

+ There are no comments

Add yours

Leave a Reply