In the race to revolutionise healthcare, artificial intelligence (AI) has taken centre stage.
From diagnosis and drug discovery to agentic hospitals (AI-powered hospitals), it promises faster, smarter, and more efficient care.
But as we marvel at its potential, it’s time we pause and ask the more complicated questions: What could go wrong? And are we ready?
1. The Illusion Of Precision
AI thrives on data. However, in healthcare, data is often incomplete, inconsistent, and biased. When algorithms are trained on datasets that don’t reflect the diversity of real-world populations—especially those from the Global South—they risk making dangerous assumptions. A skin cancer detection tool trained mostly on lighter skin tones may fail to detect conditions in darker skin. The illusion of precision becomes a dangerous trap.
What Can We Do?
We must strive to ensure that AI models used in healthcare are trained on data that represents everybody to eliminate bias. It’s a Herculean task, but if we truly want AI to work for everyone, then it’s something worth pursuing.
2. Loss Of Human Touch
Healthcare isn’t just about diagnosis and treatment. It’s about empathy, trust, companionship and communication. AI might answer your symptoms at midnight, but it can’t hold your hand in fear or explain a life-changing diagnosis with compassion. As we automate more, there’s a real danger we dehumanise care—turning patients into data points and doctors into data managers.
What Can We Do?
Instead of thinking about AI as “replacing”, why can’t we think about it as “augmenting”? The idea that AI would replace doctors, nurses, or healthcare professionals is ridiculous. Healthcare is not just about data, processes, equipment and treatments; it’s deeper and thrives on genuine human connections. We want people who can wear our shoes, hold our hands and listen to us. AI will never be able to do that. Not in a million light years. So, we think of AI as helping us do mundane tasks while we focus on the soul of healthcare—empathy, trust, companionship and communication.
FUTURE-AI Guideline, An International Consensus Framework For Trustworthy And Deployable Artificial Intelligence (AI) In Healthcare
3. Black Box Medicine
Many AI tools operate as “black boxes”—they produce results, but no one really understands how they got there. For healthcare professionals, this can erode trust and make it difficult to challenge or explain decisions. In critical situations, would you stake a patient’s life on a decision you can’t trace or justify?
What Can We Do?
Be more rigorous with the data we train AI, especially in delicate ecosystems like healthcare, drug discovery and other areas where we can’t afford to make “small mistakes”. We shouldn’t rush to integrate AI into the delicate fabric of healthcare. We must take our time and ensure that AI is pure and ready enough to be a reliable partner in caring for human beings.
4. Inequality Amplifier
AI is often touted as a leveller, but without careful design and governance, it could deepen divides. Wealthier hospitals can afford cutting-edge tools; rural or underfunded facilities cannot. Poorer communities may become testing grounds for unregulated solutions, further widening the gap between who gets the best care and who doesn’t.
What Can We Do?
Make AI technologies available to everyone, like the internet. The internet came to us free. You only needed a device and an internet connection. As we build the fundamental building blocks for AI in the future, the culture we need to adopt is one that provides everyone with equal access, at least to the basic capabilities of AI, especially in healthcare.
5. Who Is Responsible When AI Fails?
Imagine a misdiagnosis by an AI system. Who is to blame? The software developer? The hospital? The doctor who used it? In a sector where accountability is crucial, this legal and ethical grey zone could delay justice and expose patients to unprecedented risks.
What Can We Do?
Build responsibility and accountability into the AI systems we develop. Before human professionals completely accept the output of AI, there must be rigorous systems in place that thoroughly investigate and critique the output of AI! This kind of process must be enforced in sectors like healthcare, where mistakes can be fatal. This places even greater responsibilities on us as humans. We can’t leave it all to AI to do everything for us while we just sit, watch and become blunt. The best AI models are just tools. They don’t have minds and can’t be held responsible for doing something wrong or even right.
Building With The Right Framework
Artificial intelligence in healthcare must be fair, universal, traceable, usable, robust, and explainable, following the framework established by the FUTURE-AI consortium in 2021.
In building AI solutions in healthcare, we must do so with our eyes wide open and our minds ever sharp.
- Invest in transparent, explainable AI.
- Prioritise inclusive datasets.
- Create ethical frameworks and regulatory guardrails.
- Keep humans in the loop—always.
AI is a powerful tool. Like a scalpel, it can save lives—or cause harm if used recklessly.
The future of healthcare depends not just on how smart our machines become but on how wise we are in deploying them.
At Care City Media, we believe in innovation with a conscience.
Let’s build a future where technology enhances, not erases, humanity in healthcare.
What are your thoughts on the role of AI in our hospitals and clinics?
Join the conversation on Substack.
