A new report has been caricatured as saying Generative AI is a failure - whereas it actually says that most ‘Enterprise’ users are simply using it badly

Don’t Believe Everything You Read

You might well have heard about a report that came out last week from MIT NANDA (Networked Agents and Decentralized AI) allegedly proclaiming that Generative AI was a busted flush and that 95% of enterprise pilots were failing.

The press, and a certain type of Linkedin ‘Expert’ were all over it. To the extent that, on August 19th, the UK’s Financial Times proclaimed that:

“US tech stocks sold off on Tuesday as warnings that the hype surrounding artificial intelligence could be overdone hit some of the year’s best-performing shares.”

Now, as Mark Twain quipped, one should "Never let the truth get in the way of a good story” but in this case I thought I better check on the veracity behind the cacophony.

And knock me down with a feather, it turns out that the report actually argues that Generative AI is a really useful tool that almost every large company is using incorrectly. Right there in the opening paragraph it states

“Just 5% of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact. This divide does not seem to be driven by model quality or regulation, but seems to be determined by approach.”

Not My Fault Guv’nor

Bad workmen are blaming their tools!

They are calling this the ‘Gen AI Divide’ - the stark gap between those generating considerable value and none at all.

And then they are very clear what the issue is:

“The core barrier to scaling is not infrastructure, regulation, or talent. It is learning. Most GenAI systems do not retain feedback, adapt to context, or improve over time.”

And the solution:

“A small group of vendors and buyers are achieving faster progress by addressing these limitations directly. Buyers who succeed demand process-specific customization and evaluate tools based on business outcomes rather than software benchmarks. They expect systems that integrate with existing processes and improve over time. Vendors meeting these expectations are securing multi-million-dollar deployments within months.”

In order to ‘address these limitations’ they go on to explain how Agentic AI systems are what is required because these can be configured with memory, adaptability, and autonomous workflow integration.

And all this is in the first three pages of the report, which you would not have thought would be too taxing for any ‘expert’ to read.

But then, writing up a report saying enterprises have an incredible new tool at their disposal if used correctly maybe isn’t as click worthy as ‘the bubble has burst - yet again the emperor has no clothes’.

The Enterprise Prerogative

Essentially, at enterprise scale, one has to ensure:

• Integration with internal systems (CRM, ERP, lease databases)
• Governance/traceability (audit trails, compliance logs)
• Continuous improvement loops tied to business outcomes.

They need embedded, auditable, adaptive systems. And off the shelf, or even lightly customised LLMs don’t provide these.

So the report emphasises that enterprises should ‘Stop building static, non-learning tools that require endless prompting’.

What’s Wrong with Custom GPTs?

So why not build Custom GPTs?

The answer is that a Custom GPT can hold static context (instructions, tone, reference docs), but they don’t adapt dynamically from user corrections or workflow usage over time. If you correct an output today, it won’t automatically do it better tomorrow unless you manually retrain or rewrite the system prompt.

Which makes them configured assistants, rather than learning systems.

Big Is Boring - But Where The Money Is

WHICH IS FINE when using them for individual productivity, and within SMEs where the flexibility and adaptability is very much a feature rather than a bug, but in large enterprises you really need ‘process machines’.

It’s noticeable in the report that they say whilst most budget is currently pointed towards front-office applications (sales & marketing, customer operations etc) most of the ROI actually comes from building agents systems to manage back-office operations. Enterprises are process factories, not havens of entrepreneurial spirit. Frankly they need boring systems to do boring jobs that currently bored humans do.