What Should We Learn? Making Medical AI Trustworthy

The health care industry may seem the ideal place to deploy artificial intelligence systems. Each medical test, doctor’s visit, and procedure is documented, and patient records are increasingly stored in electronic formats. AI systems could digest that data and draw conclusions about how to provide better and more cost-effective care. Plenty of researchers are building such systems: Medical and computer science journals are full of articles describing experimental AIs that can parse records, scan images, and produce diagnoses and predictions about patients’ health. However, few—if any—of these systems have made their way into hospitals and clinics.

So what’s the holdup? It’s not technical, says Shinjini Kundu, a medical researcher and physician at the University of Pittsburgh School of Medicine. “The barrier is the trust aspect,” she says. “You may have a technology that works, but how do you get humans to use it and rely on it?”

Most medical AI systems operate as “black boxes” that take in data and spit out answers. Doctors are understandably wary about basing treatments on reasoning they don’t understand, so researchers are trying a variety of techniques to create systems that show their work. The paper Making Medical AI Trustworthy published by IEEE Spectrum in August 2018 give us some points on Researchers works trying to crack open the black box of AI so it can be deployed in health care.