Why AI Solutions Fail

Organisations often approach AI deployment as a software integration exercise.

Acquire a promising tool. Connect it to available data. Place it inside an existing workflow. Train staff. Launch a pilot. Then wait for improvements in efficiency, accuracy or customer experience.

But AI is not portable in quite the same way as conventional software.

An AI system arrives with assumptions about the environment in which it will operate: what data exists, which languages users speak, how decisions are made, what infrastructure is available and what happens when the system gets something wrong.

When those assumptions do not hold, a model that performs impressively in a controlled demonstration can struggle in practice.

The most consequential aspect of an AI system may therefore not be the model itself. It may be the assumptions.

The model is only one part of the system

According to Mastercard, investment in AI across Africa is accelerating across sectors. This creates opportunity, but also a risk: evaluating the model while underestimating the wider system it must enter.

A technically capable model is not, by itself, a viable product. The real system also includes data, infrastructure, workflows, human judgment, and governance.

In our work across health, agriculture and digital services, teams often focus first on what an AI tool can produce: can it diagnose, predict, recommend or automate?

Successful deployment requires other questions. Who provides the input? What happens when information is incomplete? Who can challenge the output? Can the organisation maintain and govern the system after the pilot?

There is no single African deployment environment

Discussions about AI in Africa often refer to “the African market” as though the continent presents one set of operating conditions. It does not.

A financial institution in Nairobi, a public hospital in Lusaka and an agricultural cooperative in Bungoma operate in very different conditions.

The relevant unit of analysis is the specific country, institution, workflow, user group and decision the AI is expected to support, not “Africa” in the abstract.

Infrastructure assumptions shape viability

Many AI products are designed around stable electricity, reliable connectivity, sufficient compute, and modern devices, treating them as defaults.

Infrastructure choices affect continuity, architecture and total cost of ownership. A tool that requires continuous cloud access may fail during interruptions; a large model may introduce latency and hosting costs, making an otherwise impressive product commercially unviable.

Smaller models, offline functionality, edge inference, hybrid systems or cloud deployment may each be appropriate. Architecture should follow operational reality, not the vendor’s ideal diagram.

Data and language shape model performance

AI performance can degrade when deployment data differs from the data used during development and testing.

That difference may involve language, demographic representation, incomplete records, inconsistent data entry, informal economic activity or local terminology.

A credit model built around formal employment may underestimate people whose economic activity is visible through informal trade or irregular income.

A health model may underperform for groups that were poorly represented during development. A language model may claim to support an African language but struggle with code‑switching, accents, or specialist vocabulary (sheng) in practice.

Language availability is not the same as language performance.

The answer is not simply to replace international datasets with “local data.” Local data can also be incomplete, biased or poorly governed. The stronger requirement is representative, fit‑for‑purpose data and local validation.

Who is represented? Who is missing? How does performance vary across groups? Who retains rights over the data once it creates value?

Organisations often discover only during field testing that the available data is less complete, less structured, or used differently than the product design assumed. By then, the team will have reworked both the product and the operating process around it. Earlier validation is considerably cheaper.

Real workflows carry knowledge the AI cannot see

Enterprise AI tools are often designed around linear processes: structured inputs enter the system, the model generates an output, and someone acts on it.

Real work is rarely so tidy.

Across many organisations, digital processes coexist with paper records, spreadsheets, WhatsApp messages and tacit human judgement.

A healthcare worker may rely on a patient’s verbal history when the formal record is incomplete. An agricultural officer may interpret a field problem through conversation and observation rather than a structured form.

These workarounds should not automatically be treated as inefficiencies to automate away. They often reveal where formal systems are incomplete and where experienced people are supplying judgment that the process has never captured.

In our research and design work, the formal workflow is rarely the whole workflow. Some of the most consequential decisions happen through informal handoffs, exceptions and human judgment that are invisible in process maps.

If an AI implementation bypasses that judgment without first understanding it, it can speed up the process while worsening the decision.

The person using the interface may not be the only person shaping the outcome. A supervisor, field agent, peer group or trusted intermediary may interpret the recommendation before anyone acts.

Sometimes, the right design unit is not the individual user. It is the decision network.

Teams must also understand the constraint AI is expected to address. Is it informational, decisional or material?

A farmer may receive accurate advice but be unable to afford the recommended input. A clinician may receive a useful risk flag but lack the referral capacity to respond.

In these cases, AI may need to sit within a wider service, financing, referral or workflow redesign. Otherwise, it risks describing the problem more efficiently without changing the outcome.

Governance determines whether the organisation remains in control

Responsible AI deployment is sometimes treated as a legal or compliance review near the end of procurement. That is too late.

Governance should shape how the system is designed, acquired, tested, operated and monitored.

Organisations need clarity on data ownership, consent, accountability, human oversight and performance monitoring. They also need the capability to evaluate the model, challenge vendor claims and change providers if necessary.

A tool that performs well but leaves the organisation permanently dependent on expertise or infrastructure it cannot control has solved only part of the deployment problem.

The question is not whether every component must be built locally. The question is whether the organisation has sufficient knowledge, authority, and control to use the system responsibly over time.

Usage is not the same as value

AI pilots are often evaluated through easy-to-collect metrics: users registered, logins, prompts, outputs generated or staff trained.

These figures may show a tool is available. They do not necessarily show that it has become part of meaningful decision‑making.

There is a difference between technical usage, workflow adoption, trusted decision influence and measurable value.

An employee may open an AI tool but continue relying on the old process. A farmer may receive advice but still consult a trusted peer before acting.

For technology leaders, the better measures are whether decision quality improved, errors declined, time was saved, and the value justified the costs of integration, oversight, and maintenance.

This is why organisations must test not only whether an AI tool can produce a technically correct output, but whether the surrounding system can use that output reliably.

A checklist before scaling AI

Taking a context‑sensitive approach to deployment is not simply about choosing global or local models. It is to refuse to transfer assumptions without testing them against real conditions – knowing what can travel, what must be adapted, and what capabilities organisations need to retain locally.

Before scaling an AI solution, leaders should ask:

What decision or capability is the AI meant to improve?
Has the system been tested with the data, languages, infrastructure, workflows and users it will encounter?
Can the organisation operate, challenge and govern it?
What evidence will demonstrate value beyond usage?

Context‑first AI is good business

This is not only an ethical or inclusion argument. It is a return‑on‑investment argument.

AI deployments that ignore context create failed pilots, rework, unreliable outputs, low adoption, vendor dependence and stranded investment.

By contrast, organisations that validate the entire deployment system are more likely to achieve reliable performance, staff acceptance, user trust and sustainable scale.

The race to shape Africa’s AI future will not be won by organisations that acquire the most sophisticated models first. It will be won by those who understand the environments in which those models must work and have the discipline to adapt the whole system accordingly.

So the question before deployment isn’t simply whether the model works. But:

Is the whole AI system working here — for this decision, with this data, through this workflow, for these people, under these constraints — and can the organisation govern it as conditions change?

Maureen Macharia is CEO of Spindle Design, a research, design and digital innovation firm helping organisations build digital products, programmes and services grounded in real user behaviour and market realities across Africa.