From Pilots to Responsible Scale: Reflections from Madrid

Why better AI in healthcare begins with real data, human systems and collective action

(Reading time of around 7–9 minutes).  

Last month, Hemangi Rajpara and I represented the Health Data Forum at the OECD International Conference on the Responsible Scale of AI in Health, hosted in Madrid in collaboration with the Spanish Ministry of Health.

The event brought together policymakers, health-system leaders, researchers, clinicians, industry representatives and civil-society voices around a question that is becoming increasingly urgent:

How can artificial intelligence move beyond promising pilots and become a responsible, sustainable and equitable part of real healthcare systems?

The word scale was central to the conference. But so was the word responsible.

That distinction matters.

Healthcare is full of impressive AI demonstrations. What remains far less common is sustained adoption across clinical pathways, institutions, regions and populations, with clear evidence of better outcomes and public value.

Madrid, therefore, felt like an important turning point in the conversation. The question is no longer only whether AI can perform a particular task. It is whether our health systems possess the foundations, capacity and legitimacy required to use it well.

The problem is not simply technological

One of the clearest messages emerging from the OECD work is that the difficulty of scaling AI in healthcare is not primarily a technology problem.

It is a systems problem.

Health organisations do not operate like technology companies. They work within complex clinical pathways, demanding regulatory environments, constrained budgets, fragmented infrastructure and deeply human relationships involving vulnerability, trust and risk.

An algorithm may demonstrate excellent results in a controlled setting and still fail to deliver meaningful value when implemented in daily care.

Why?

Because implementation requires much more than software.

It requires usable data, interoperable infrastructure, workforce confidence, workflow redesign, governance, procurement, evaluation, accountability and long-term organisational commitment.

AI cannot simply be placed atop an unprepared health system and expected to transform it.

Without these foundations, costs increase as solutions move towards deployment, while the value they create may quickly plateau. The result is a growing gap between technological promise and system-level impact.

The end of "pilotitis"

The conference repeatedly returned to a familiar pattern: successful pilots that never progress into sustained adoption.

This is sometimes described as pilotitis.

Pilotitis is not a failure of imagination. It is often a failure to design for scale from the beginning.

A pilot can be technically successful while avoiding the hardest questions:

• Who is accountable when the system is deployed?

• How will it fit into existing clinical workflows?

• Is the workforce prepared to use it?

• Can it operate across different institutions and populations?

• How will its performance be monitored over time?

• Who pays for implementation, maintenance and updating?

• What happens when the data changes?

• How will patients and professionals be involved?

The true test of AI in health begins where the demonstration ends.

It begins in the real world: with incomplete data, complex patients, overstretched professionals, organisational boundaries and competing priorities.

Responsible scale must therefore be designed into innovation from the outset. It cannot be added later as an afterthought in the implementation.

Real Data, Better AI

The discussions in Madrid strongly reinforced the evolution of our own movement.

When we introduced Data First, AI Later, the message was intentionally direct: healthcare should not rush towards artificial intelligence without first addressing the quality, integrity, governance and usability of the data on which it depends.

That principle remains as relevant as ever.

But the movement is now entering a new stage under the name Real Data, Better AI.

This is not a rejection of the original idea. It is its natural development.

"Real data" does not merely mean data collected in real-world settings. It also means data that reflects the complexity, diversity and context of real patients and health systems.

It means data that is:

• reliable enough to support decisions;

• representative enough to reduce bias;

• interoperable enough to travel across systems;

• governed well enough to earn trust;

• understandable enough to be used in practice;

• and connected to outcomes that matter to patients and society.

Better AI cannot be separated from these conditions.

An AI system trained on fragmented, biased or poorly governed data will not overcome those weaknesses. It may amplify them.

The quality of our intelligence will depend on the quality of our foundations.

Three connected responsibilities

The emerging Madrid Action Plan organises responsible scale around three broad responsibilities:

Earn trust

Trust cannot be treated as a communications campaign introduced after deployment.

It must be embedded throughout the AI lifecycle.

This includes bioethics, transparency, explainability, patient and professional involvement, workforce capability and clear communication about limitations and risk.

At the Health Data Forum Global Hybrid Summit in Wales, we reached a similar conclusion: trust is not a transaction; it is a relationship.

It begins before consent, evolves throughout the use of data and must be maintained after that data has contributed to research, innovation or care.

People need to know not only that their data is protected, but also why it is being used, by whom, with what safeguards and for whose benefit.

Enable what works

Health systems need to identify, evaluate and scale solutions that respond to genuine health and care needs.

This requires trusted data collaboration, interoperable technical infrastructure, aligned regulatory and evaluation frameworks, sustainable financial models and procurement connected to health-system goals.

It also requires an important shift away from technology-led innovation.

The starting question should not be:

Where can we deploy this tool?

It should be:

What problem are patients, professionals and health systems trying to solve?

Only then should we determine whether AI is the right instrument and what conditions are required for it to succeed.

Prevent harm

Healthcare cannot move responsibly at scale without safeguards.

Privacy and cybersecurity must be designed into systems from the beginning. Human oversight must remain meaningful. AI uncertainties, limitations and potential biases need to be identified, communicated and monitored before and after deployment.

This also means recognising that harm can come from both action and inaction.

Overly rigid systems may delay beneficial innovation, while careless deployment may expose patients and organisations to unacceptable risks.

Responsible scale requires proportionate judgement rather than either blind acceleration or permanent paralysis.

The workforce cannot be an afterthought

One of my strongest reflections from Madrid concerns the health workforce.

Professionals are frequently told that AI will reduce their burden. Yet many have experienced previous waves of digitalisation as the opposite: more screens, more documentation, more fragmented systems and more work outside normal hours.

Scepticism is therefore understandable.

Trust among professionals will not be created through promises. It will be built through participation, evidence and practical experience.

Healthcare workers must be involved in the design, evaluation and implementation of AI tools. They need time, support and relevant education—not only technical training, but also the confidence to question outputs, understand limitations and recognise when human judgement must prevail.

The workforce is not a barrier to innovation.

It is the essential bridge between technology and better care.

The process matters as much as the content

As a professional facilitator, I was also struck by the design of the Madrid meeting itself.

The organisers did not treat responsible AI as a subject that could be addressed only through speeches from a stage. The conference created structured opportunities for dialogue, breakout work, collective prioritisation and relationship-building.

That matters because a complex transformation cannot be produced through content alone.

It depends on the process.

The distinction between content and process is fundamental in any meaningful meeting. Content concerns what we discuss. Process determines how people participate, how different forms of knowledge are included, how tensions are handled and whether the dialogue produces ownership.

The meeting also included a cultural dimension through a private museum visit and shared dinner. This was not peripheral to the work. It reminded participants that healthcare, technology and public policy ultimately exist within human life, culture and relationships.

The responsible adoption of AI must preserve that same sense of humanity.

From Madrid to Bilbao

The Madrid conversation also connects directly with the next Health Data Forum Global Hybrid Summit, taking place in Bilbao on 24 and 25 September 2026.

Madrid addressed the challenge of responsible scale at the system level.

Bilbao will take that challenge further into implementation.

Our programme will explore how the European Health Data Space can work in real-world settings, with particular attention to governance, trusted secondary use, interoperability, cybersecurity, genomics, medical imaging, infrastructure and measurable health-system value.

The goal is not to celebrate AI in the abstract.

It is about understanding what must be built, aligned, and governed so that innovation can move responsibly from policy to practice.

Madrid offered an important international framework. Bilbao can become a practical laboratory for the next stage of the conversation.

A collective challenge

No organisation, government or technology provider can solve this challenge alone.

Responsible AI in health depends on collaboration across sectors and borders. It requires public institutions, clinicians, patients, researchers, regulators, industry and civil society to share responsibility for the systems being created.

It also requires a different understanding of innovation.

Innovation is not simply the invention of something new.

In healthcare, innovation only becomes meaningful when it can be integrated safely, trusted by those affected, sustained financially and translated into better outcomes.

The future of AI in health will not be determined solely by the power of our models.

It will be determined by the quality of our data, the strength of our institutions, the preparedness of our workforce, the legitimacy of our governance and our capacity to collaborate.

That is the promise—and the responsibility—behind Real Data, Better AI.

The journey from pilots to responsible scale has begun.