Skip to main content

AI in Medical Imaging Could Magnify Health Inequities, Study Finds

May 02, 2023 | Vanessa McMains

Addressing Biases Now and Promoting Awareness Can Encourage Fairness in Medicine

Artificial intelligence (AI) technology in the medical field has the possibility to automate diagnoses, decrease physician workload, and even to bring specialized healthcare to people in rural areas or developing countries. However, with possibility comes potential pitfalls.

Paul Yi , MDAnalyzing crowd-sourced sets of data used to create AI algorithms from medical images, University of Maryland School of Medicine (UMSOM) researchers found that most did not include patient demographics. In the study published April 3 in Nature Medicine, the researchers also found that the algorithms did not evaluate for inherent biases either. That means they have no way of knowing whether these images contain representative samples of the population such as Blacks, Asians, and Indigenous Americans.

According to the researchers, much of medicine in the U.S. is already fraught with partiality toward certain races, genders, ages, or sexual orientations. Small biases in individual sets of data could be amplified greatly when hundreds or thousands of these datasets are combined in these algorithms.

“These deep learning models can diagnose things physicians can’t see, such as when a person might die or detect Alzheimer’s disease seven years earlier than our known tests superhuman tasks,” said senior investigator Paul Yi, MD, Assistant Professor of Diagnostic Radiology and Nuclear Medicine at UMSOM. He is also Director of University of Maryland Medical Intelligent Imaging (UM2ii) Center. Because these AI machine learning techniques are so good at finding needles in a haystack, they can also define sex, gender, and age, which means these models can then use those features to make biased decisions.”

Much of the data collected in large studies tends to be from people of means who have relatively easy access to healthcare. In the U.S., this means the data tends to be skewed toward men versus women, and toward people who are white rather than other races. As the U.S. tends to perform more imaging than the rest of the world, this data gets compiled into algorithms that have the potential to slant outcomes worldwide.

For the current study, the researchers chose to evaluate the datasets used in data science competitions in which computer scientists and physicians crowdsource data from around the world and try to develop the best, most accurate algorithm. These competitions tend to have leaderboards that rank each algorithm and provide a cash prize, motivating people to create the best one. Specifically, the researchers investigated medical imaging algorithms, such as those that evaluate CT scans to diagnose brain tumors or blood clots in the lungs. Of the 23 data competitions analyzed, 61 percent did not include demographic data such as age, sex, or race. None of the competitions had evaluations for biases against underrepresented or disadvantaged groups.

Mark T. Gladwin, MD“We hope that by bringing awareness to this issue in these data competitions – and if applied in an appropriate way – that there is tremendous potential to solve these biases,” said lead author Sean Garin, Program Coordinator at the UM2ii Center.

The study’s authors also encourage future competitions to require not only high accuracy, but also fairness among different groups of people.

“As AI models become more prevalent in medical imaging and other fields of medicine, it is important to identify and address potential biases that may exacerbate existing health inequities in clinical care – an essential priority for every academic medical institution,” said UMSOM Dean Mark T. Gladwin, MD, Vice President for Medical Affairs, University of Maryland, Baltimore, and the John Z. and Akiko K. Bowers Distinguished Professor. 

About the University of Maryland School of Medicine

Now in its third century, the University of Maryland School of Medicine was chartered in 1807 as the first public medical school in the United States. It continues today as one of the fastest growing, top-tier biomedical research enterprises in the world — with 46 academic departments, centers, institutes, and programs, and a faculty of more than 3,000 physicians, scientists, and allied health professionals, including members of the National Academy of Medicine and the National Academy of Sciences, and a distinguished two-time winner of the Albert E. Lasker Award in Medical Research. With an operating budget of more than $1.3 billion, the School of Medicine works closely in partnership with the University of Maryland Medical Center and Medical System to provide research-intensive, academic, and clinically based care for nearly 2 million patients each year. The School of Medicine has nearly $600 million in extramural funding, with most of its academic departments highly ranked among all medical schools in the nation in research funding. As one of the seven professional schools that make up the University of Maryland, Baltimore campus, the School of Medicine has a total population of nearly 9,000 faculty and staff, including 2,500 students, trainees, residents, and fellows. The combined School of Medicine and Medical System (“University of Maryland Medicine”) has an annual budget of over $6 billion and an economic impact of nearly $20 billion on the state and local community. The School of Medicine, which ranks as the 8th highest among public medical schools in research productivity (according to the Association of American Medical Colleges profile) is an innovator in translational medicine, with 606 active patents and 52 start-up companies. In the latest U.S. News & World Report ranking of the Best Medical Schools, published in 2021, the UM School of Medicine is ranked #9 among the 92 public medical schools in the U.S., and in the top 15 percent (#27) of all 192 public and private U.S. medical schools. The School of Medicine works locally, nationally, and globally, with research and treatment facilities in 36 countries around the world. Visit


Vanessa McMains
Director, Media & Public Affairs
University of Maryland School of Medicine
Institute of Human Virology
Cell: 443-875-6099

Related stories

    Wednesday, April 26, 2023

    Long COVID Sufferers with Cognitive Complaints and Mental Health Issues Have Changes in Brain Function, New Study Suggests

    Those who experience brain fog, memory issues or other neuropsychiatric symptoms for months after a COVID-19 diagnosis were found to have abnormal brain activity during memory tests on functional magnetic resonance imaging (MRI), according to a new study led by University of Maryland School of Medicine (UMSOM) researchers.

    Friday, April 21, 2023

    In Memoriam: Rao P. Gullapalli, PhD, MBA

    The University of Maryland School of Medicine community is mourning the loss of one of its longtime senior faculty members, Rao P. Gullapalli, PhD, MBA, who was Professor of Diagnostic Radiology and Nuclear Medicine, Associate Vice Chair for Research in the Department and Director of the Core for Translational Research in Imaging (C-TRIM) and Director of the Center for Metabolic Imaging and Therapeutics (CMIT).  He also held several leadership roles in cancer imaging at the UM Marlene and Stewart Greenebaum Comprehensive Cancer Center.

    Tuesday, April 04, 2023

    UM School of Medicine Study Finds ChatGPT Helpful for Breast Cancer Screening Advice, With Certain Caveats

    As more consumers turn to the newly available ChatGPT for health advice, researchers are eager to see whether the information provided by the artificial intelligence chatbot is reliable and accurate. A new study conducted by researchers at the University of Maryland School of Medicine (UMSOM) indicates that the answers generated provide correct information the vast majority of the time; sometimes, though, the information is inaccurate or even fictitious.

    Friday, July 29, 2022

    Children Who Lack Sleep May Experience Detrimental Impact on Brain and Cognitive Development That Persists Over Time, UM School of Medicine Study Finds

    Elementary school-age children who get less than nine hours of sleep per night have significant differences in certain brain regions responsible for memory, intelligence, and well-being compared to those who get the recommended 9-12 hours of sleep per night, according to a new study led by University of Maryland School of Medicine (UMSOM) researchers. Such differences correlated with greater mental health problems like depression, anxiety, and impulsive behaviors in those who lacked sleep. Inadequate sleep was also linked to cognitive difficulties with memory, problem solving and decision making. The findings were published today in the journal Lancet Child & Adolescent Health.

    Monday, January 31, 2022

    New Technique Consistently Works to Open Blood-Brain Barrier, Paving Way for Pre-Clinical Studies of Neurologic Drugs

    University of Maryland School of Medicine researchers developed a technique in laboratory animals to consistently and reproducibly open the blood-brain barrier. This barrier serves as a barricade securing the brain from the external world blocking out certain environmental toxins, but also prevents drug therapies from reaching their intended targets. The new technique is based on a routine procedure for removing clots from the brain’s arteries in patients. This advancement was conducted by Piotr Walczak, MD, PhD, Professor of Diagnostic Radiology and Nuclear Medicine at the University of Maryland School of Medicine, and Miroslaw Janowski, MD, PhD, Associate Professor of Diagnostic Radiology and Nuclear Medicine at the University of Maryland School of Medicine.