Skip to content
Search

Latest Stories

Follow Us:
Top Stories

AI could help remove bias from medical research and data

Opinion

Researcher looks at mammography test

Artificial intelligence can help root out racial bias in health care, but only if the programmers can create the software so it doesn't make the same mistakes people make, like misreading mammograms results, writes Pearl.

Anne-Christine Poujoulat/AFP via Getty Images
Pearl is a clinical professor of plastic surgery at the Stanford University School of Medicine and is on the faculty of the Stanford Graduate School of Business. He is a former CEO of The Permanente Medical Group.

This is the second entry in a two-part op-ed series on institutional racism in American medicine.

A little over a year before the coronavirus pandemic reached our shores, the racism problem in U.S. health care was making big headlines.

But it wasn't doctors or nurses being accused of bias. Rather, a study published in Science concluded that a predictive health care algorithm had, itself, discriminated against Black patients.

The story originated with Optum, a subsidiary of insurance giant UnitedHealth Group, which had designed an application to identify high-risk patients with untreated chronic diseases. The company's ultimate goal was to help re-distribute medical resources to those who'd benefit most from added care. And to figure out who was most in need, Optum's algorithm assessed the cost of each patient's past treatments.

Unaccounted for in the algorithm's design was this essential fact: The average Black patient receives $1,800 less per year in total medical care than a white person with the same set of health problems. And, sure enough, when the researchers went back and re-ranked patients by their illnesses (rather than the cost of their care), the percentage of Black patients who should have been enrolled in specialized care programs jumped from 18 percent to 47 percent.

Journalists and commentators pinned the blame for racial bias on Optum's algorithm. In reality, technology wasn't the problem. At issue were the doctors who failed to provide sufficient medical care to the Black patients in the first place. Meaning, the data was faulty because humans failed to provide equitable care.

Artificial intelligence and algorithmic approaches can only be as accurate, reliable and helpful as the data they're given. If the human inputs are unreliable, the data will be, as well.

Let's use the identification of breast cancer as an example. As much as one-third of the time, two radiologists looking at the same mammogram will disagree on the diagnosis. Therefore, if AI software were programmed to act like humans, the technology would be wrong one-third of the time.

Instead, AI can store and compare tens of thousands of mammogram images — comparing examples of women with cancer and without — to detect hundreds of subtle differences that humans often overlook. It can remember all those tiny differences when reviewing new mammograms, which is why AI is already estimated to be 10 percent more accurate than the average radiologist.

What AI can't recognize is whether it's being fed biased or incorrect information. Adjusting for bias in research and data aggregation requires that humans acknowledge their faulty assumptions and decisions, and then modify the inputs accordingly.

Correcting these types of errors should be standard practice by now. After all, any research project that seeks funding and publication is required to include an analysis of potential bias, based on the study's participants. As an example, investigators who want to compare people's health in two cities would be required to modify the study's design if they failed to account for major differences in age, education or other factors that might inappropriately tilt the results.

Given how often data is flawed, the possibility of racial bias should be explicitly factored into every AI project. With universities and funding agencies increasingly focused on racial issues in medicine, this expectation has the potential to become routine in the future. Once it is, AI will force researchers to confront bias in health care. As a result, the conclusions and recommendations they provide will be more accurate and equitable.

Thirteen months into the pandemic, Covid-19 continues to kill Black individuals at a rate three times higher than white people. For years, health plans and hospital leaders have talked about the need to address health disparities like these. And yet, despite good intentions, the solutions they put forth always look a lot like the failed efforts of the past.

Addressing systemic racism in medicine requires that we analyze far more data (all at once) than we do today. AI is the perfect application for this task. What we need is a national commitment to use these types of technologies to answer medicine's most urgent questions.

There is no antidote to the problem of racism in medicine. But combining AI with a national commitment to root out bias in health care would be a good start, putting our medical system on a path toward antiracism.


Read More

unrecognizable businessman giving speech to professional audience

Brain science explains why voters reward charisma over results — and why the leaders who "fix" the most get credit for the least. A behavioral look at elections.

alvaro gonzalez/Getty Images

The Résumé Should Outvote the Smile

There's an old joke about a couple who finally solved years of arguing over decisions. "Since you're the man of the house," the wife said, "you handle the big things -- our stance on foreign wars, who we send to the Senate. I'll take care of the small stuff: groceries, the mortgage, where we live, what we eat." The husband agreed happily, certain he had won the better half of the deal. A few years later he realized the trade had gone entirely the other way. He held forth on geopolitics at dinner parties while his wife quietly ran every detail that actually touched his life: what he ate, where he lived, how his money was spent. What looked like ultimate authority was, in practice, near-total abdication over the things that shaped his days.

Voters make the same trade, and pay for it the same way. We keep hiring leaders for the qualities we would want in a spouse -- warmth, charm, a commanding voice, the sense that they see us -- instead of the qualities we would want in a mechanic: someone who can find what is broken and fix it before the whole system fails. We evaluate candidates the way we choose intimate partners, not the way we hire people to run complex, high-stakes systems.

Keep ReadingShow less
US soldier with American flag patch on his arm.
a group of soldiers in uniform
Photo by Oleg Ivanov on Unsplash

Moral Leadership Can Protect Service Members' Psychological Well-Being

It has recently been reported that President Trump and Secretary of War Hegseth disparaged the mental health concerns, including suicide attempts, expressed by the sailors serving aboard the USS Abraham Lincoln during the carrier’s record deployment. Their comments brought back the disquiet I felt several months ago while attending a performance of the US Naval Academy Glee Club at the Church of St. Ignatius Loyola in Manhattan. Usually, when I attend choral concerts in the church’s imposing Baroque sanctuary, my mind quiets, and I find myself immersed in the music. But not on that occasion. Instead, as I enjoyed the exuberant singing, my mind filled with resentment. The Glee Club, handsome midshipmen in formal dress, is a multiethnic group of men and women. The harmonious singing of a chorus composed of individuals of widely varied backgrounds seemed a metaphor for how our country and its military should work.

My upset originated with the realization that during their time at the Naval Academy, these midshipmen had witnessed the firing of the Black Chairman of the Joint Chiefs of Staff, Air Force General C. Q. Brown, Jr., as part of an apparent purge of those supporting diversity in the military. Even uglier was the implication that Brown, a highly decorated officer, had been elevated to his position as a ‘DEI’ hire. They also had seen the firing of trailblazing women leaders in the military, including Admiral Linda Fagan, the first woman Coast Guard Commandant, and Admiral Yvette Davids, the first woman Superintendent of the Naval Academy at Annapolis, the very institution they were attending. I was offended and angered by the message these firings conveyed to the men and women of the Academy’s Glee Club, especially the women and minority midshipmen, who had volunteered to serve their nation and, ultimately, to place themselves in harm’s way for it.

Keep ReadingShow less
Donald Trump

U.S. President Donald Trump speaks during a“ Steel Across America” event ahead of the 25th anniversary of the Sept. 11th attacks on the Ellipse near the White House on Sept. 8, 2026 in Washington, D.C. President Trump welcomed 9/ 11 first responders to the Tunnel to Towers Foundation’ s“ Steel Across America” event featuring a 16,900- pound, 21- foot steel beam recovered.

(Getty Images)

For Trump, even 9/11 is an opportunity

There was a reason organizers of the 9/11 ceremony in Manhattan told President Trump he could not speak at this year’s memorial service.

A good one, it turns out.

Keep ReadingShow less
Donald Trump

Donald Trump, September 09, 2026

Getty Images

Trump, Kennedy, and the Monkey’s Paw

The hardest deal to judge is one you helped sell.

In 2024, I served as North Carolina state director for Robert F. Kennedy Jr.’s presidential campaign and co-founded We The People of North Carolina, the independent party that put him on our state’s ballot.

Keep ReadingShow less