Research findings from MIT recently went viral with a sensational claim that 95% of artificial intelligence pilot projects have no real impact on the bottom line.
Independent observers have widely questioned the findings, which don’t align with my own experience at Datacom, where we have seen customers achieve significant productivity gains through the use of AI. Indeed, Datacom’s 2025 State of AI Index found that 88% of organisations report a positive impact on operations from the use of AI.
But it also revealed that just 8 percent of respondents identified their organisation as using AI to transform their core operations, while 46 percent described their AI use as being in the exploratory phase. It mirrors other study findings that suggest an initial wave of experimentation with generative AI systems drawing on large language models has run out of steam.
This is not an inherent problem with AI itself. Rather, as the Harvard Business School management professors Nathan Furr and Andrew Shipilov suggest, the problem with failed AI projects is an age-old one. This has been repeated throughout history when technology has been developed without a clear connection to the real business opportunity.
“We believe that many leaders are making the same mistake they made a decade earlier with digital transformation: encouraging experimentation, which is good, but falling into the trap of letting experimentation run wild, which is counterproductive.”
Datacom’s own Director of AI, Lou Compagnone, has cautioned against getting stuck in the proof-of-concept phase or “POC purgatory”.
Ensuring that AI experiments or proof-of-concept projects are set up in a way that allows them to scale up and enable the business is now the priority for any organisation looking to deploy AI.
But I’d argue that there’s a more fundamental issue that remains overlooked. Often, the underlying data feeding AI systems is simply not fit for purpose.
In the rush to harness AI’s potential, organisations too often overlook the foundational work of preparing, shaping and governing their data. The allure of quick wins, plugging a language model into a patchwork of data sources or letting generative tools loose on poorly indexed files lacking appropriate metadata, can lead to expensive failures and dashed expectations.
Simply having a data lake or data warehouse and depositing reams of unstructured and structured information is not enough. AI models are only as smart as the data that feeds them and mismatched or low-quality data will amplify noise, bias and error. The 2025 State of AI Index bears this out, with concerns over the quality of data identified as a key barrier to scaling up AI projects.
At Datacom, our mantra to clients is simple: get your data house in order before dreaming big with AI. Too many business leaders sign off on digital transformation programs, building data lakes or deploying new warehouses and expect AI to simply sort through the mess. But AI doesn’t magically compensate for missing, inaccurate or stale data. In fact, it magnifies those issues, potentially driving poor decision-making and regulatory headaches.
Many organisations across Australia and New Zealand have made significant progress in preparing their data over the last decade, which is helping them benefit greatly from data analytics, business intelligence and data-driven decision-making.
But dirty, incomplete, or irrelevant data still sidetracks even some of the most promising AI initiatives. The challenge is compounded by “shadow AI”, enthusiastic but unsanctioned experiments with generative tools by staff (and even customers), often using sensitive or poorly managed datasets. While these efforts are innovative, they rarely scale safely or produce results a business can trust with its reputation and resources.
What propels AI projects to success? Through years of helping organisations on their data and AI journey, I’ve seen the following principles repeatedly make a big difference:
Avoid jumping into ambitious, organisation-wide AI projects before demonstrating value in a specific, controllable area of your business. Focus on well-understood processes with available, clean data, where gains can be measured and lessons learned quickly. Success in one area builds confidence and culture for broader adoption. (Scaling AI across your organisation is a critical step, but it’s not a first step.)
Include the people who live day-to-day with the problems. They know where the pain points are and can help define what success truly looks like. Be prepared to “fail fast”. Test, refine and validate, moving quickly when something works and pivoting when it doesn’t.
Clean, consistent, “fit for purpose” data is non-negotiable. Invest in cataloguing data, establishing policies for collection, maintenance and access and embedding strong governance mechanisms. Metadata, which helps the AI understand what each data set represents and how it is allowed to be used, makes downstream AI efforts safer and more effective.
The AI landscape is evolving at a breakneck pace. Today’s custom build may be tomorrow’s commodity feature. Set clear boundaries, know where AI will (and won't) be applied and be pragmatic about what not to automate, at least for now.
Achieving AI return on investment (ROI) is not about adopting the flashiest technology. It’s about building solid data foundations. Most New Zealand businesses don’t have armies of data or AI scientists or unlimited budgets, so learn from others’ mistakes. Don’t hesitate to seek external help. We can steer you clear of common pitfalls – we've seen the good, the bad and the ugly.
Before jumping into AI, look long and hard at the state of your data. The best AI strategy starts with getting your house in order, because quality data is the bedrock on which every successful AI project is built.
Many AI projects fail not because of the technology but due to poor data foundations. At Datacom, we help New Zealand businesses turn raw data into actionable insights that guide smarter technology decisions. With advanced analytics, dashboards and usage reports, we deliver visibility into asset performance, spend and procurement. This ensures cost efficiencies, risk reduction and smarter planning for long-term success.
