Skip to content
Search

Latest Stories

Follow Us:
Top Stories

AI could help remove bias from medical research and data

Opinion

Researcher looks at mammography test

Artificial intelligence can help root out racial bias in health care, but only if the programmers can create the software so it doesn't make the same mistakes people make, like misreading mammograms results, writes Pearl.

Anne-Christine Poujoulat/AFP via Getty Images
Pearl is a clinical professor of plastic surgery at the Stanford University School of Medicine and is on the faculty of the Stanford Graduate School of Business. He is a former CEO of The Permanente Medical Group.

This is the second entry in a two-part op-ed series on institutional racism in American medicine.

A little over a year before the coronavirus pandemic reached our shores, the racism problem in U.S. health care was making big headlines.

But it wasn't doctors or nurses being accused of bias. Rather, a study published in Science concluded that a predictive health care algorithm had, itself, discriminated against Black patients.

The story originated with Optum, a subsidiary of insurance giant UnitedHealth Group, which had designed an application to identify high-risk patients with untreated chronic diseases. The company's ultimate goal was to help re-distribute medical resources to those who'd benefit most from added care. And to figure out who was most in need, Optum's algorithm assessed the cost of each patient's past treatments.

Unaccounted for in the algorithm's design was this essential fact: The average Black patient receives $1,800 less per year in total medical care than a white person with the same set of health problems. And, sure enough, when the researchers went back and re-ranked patients by their illnesses (rather than the cost of their care), the percentage of Black patients who should have been enrolled in specialized care programs jumped from 18 percent to 47 percent.

Journalists and commentators pinned the blame for racial bias on Optum's algorithm. In reality, technology wasn't the problem. At issue were the doctors who failed to provide sufficient medical care to the Black patients in the first place. Meaning, the data was faulty because humans failed to provide equitable care.

Artificial intelligence and algorithmic approaches can only be as accurate, reliable and helpful as the data they're given. If the human inputs are unreliable, the data will be, as well.

Let's use the identification of breast cancer as an example. As much as one-third of the time, two radiologists looking at the same mammogram will disagree on the diagnosis. Therefore, if AI software were programmed to act like humans, the technology would be wrong one-third of the time.

Instead, AI can store and compare tens of thousands of mammogram images — comparing examples of women with cancer and without — to detect hundreds of subtle differences that humans often overlook. It can remember all those tiny differences when reviewing new mammograms, which is why AI is already estimated to be 10 percent more accurate than the average radiologist.

What AI can't recognize is whether it's being fed biased or incorrect information. Adjusting for bias in research and data aggregation requires that humans acknowledge their faulty assumptions and decisions, and then modify the inputs accordingly.

Correcting these types of errors should be standard practice by now. After all, any research project that seeks funding and publication is required to include an analysis of potential bias, based on the study's participants. As an example, investigators who want to compare people's health in two cities would be required to modify the study's design if they failed to account for major differences in age, education or other factors that might inappropriately tilt the results.

Given how often data is flawed, the possibility of racial bias should be explicitly factored into every AI project. With universities and funding agencies increasingly focused on racial issues in medicine, this expectation has the potential to become routine in the future. Once it is, AI will force researchers to confront bias in health care. As a result, the conclusions and recommendations they provide will be more accurate and equitable.

Thirteen months into the pandemic, Covid-19 continues to kill Black individuals at a rate three times higher than white people. For years, health plans and hospital leaders have talked about the need to address health disparities like these. And yet, despite good intentions, the solutions they put forth always look a lot like the failed efforts of the past.

Addressing systemic racism in medicine requires that we analyze far more data (all at once) than we do today. AI is the perfect application for this task. What we need is a national commitment to use these types of technologies to answer medicine's most urgent questions.

There is no antidote to the problem of racism in medicine. But combining AI with a national commitment to root out bias in health care would be a good start, putting our medical system on a path toward antiracism.


Read More

Trump’s second term is a murky, embarrassing and costly spectacle

U.S. President Donald Trump displays a graph entitled "Our Pool is Bigger than Skyscrapers" as he speaks on his renovations to the Lincoln Memorial Reflecting Pool during an event in the Oval Office of the White House on June 3, 2026, in Washington, D.C.

(Kevin Dietsch/Getty Images/TNS)

Trump’s second term is a murky, embarrassing and costly spectacle

Every time I get asked by a TV anchor what I think about the drama of the Lincoln Memorial Reflecting Pool, my favorite “historical” headline from the Onion comes to mind: “World’s Largest Metaphor Hits Ice-Berg.”

And every time I do, I hear from defenders of the Trump administration complaining about the disproportionate media coverage of what should be a very minor story in the grand sweep of things. They have a point. President Trump has done some good work rehabbing Washington, D.C., where I live. But the Reflecting Pool has bedeviled him. Algae keep returning to the pool, despite the administration’s best efforts, and attempts to remedy the problem have yielded further problems.

Keep ReadingShow less
Only Trump doesn’t care about housing

A view of the U.S. Capitol in Washington, D.C., on June 25, 2026. President Donald Trump jolted Republicans during a fiery appearance at the U.S. Capitol on Wednesday, scrapping a housing bill signing ceremony and clashing behind closed doors with a party rebel who challenged him over the Iran war. Trump had been expected to sign the bipartisan housing.

(AFP via Getty Images)

Only Trump doesn’t care about housing

It was August 15, 2024. Then candidate Donald Trump stepped out of his Bedminster, New Jersey, golf club’s columned clubhouse to a gaggle of reporters. He was flanked by tables of groceries and signs showing the rising cost of food. Also on one of the tables was a dollhouse, meant to represent the equally alarming rise in housing prices.

It was a speech about the economy, the single most important issue of the 2024 election cycle, full of promises that went right to the heart of Americans’ anxieties. While former President Joe Biden and then Vice President Kamala Harris were contorting themselves to posture a good economy that just needed more time to recover from the pandemic, Trump was preying on voters’ very real fears of unaffordable gas, groceries, and homes. It was obviously a winning message.

Keep ReadingShow less
July 4th and the American Faith We’ve Watched Slip Away

Kids and families celebrate the US Bicentennial near the New York Harbor in Lower Manhattan. Taken on July 4, 1976 in New York City, New York.

(Photo by David Attie/Getty Images.)

July 4th and the American Faith We’ve Watched Slip Away

I was a girl in Philadelphia in the summer when America turned 200. The birthplace of America was electric in a way I've never forgotten — crowds stretching from the art museum steps down to the Delaware River, each city block corded off for parades, cookouts, celebrations, and the kind of noise that felt like belonging.

It was also, I know now, a particular kind of American moment — one that required something beyond good weather and a long weekend. It required a belief that the country and its highest office still belonged to all of us.

Keep ReadingShow less
Protestors holding flags that read, "Trump 2020," and recording on their phones inside the U.S. Capitol.

A pro-Trump mob enters the Rotunda of the U.S. Capitol Building on January 6, 2021 in Washington, DC. Congress held a joint session today to ratify President-elect Joe Biden's 306-232 Electoral College win over President Donald Trump.

Win McNamee / Getty Images

MAGA’s Get Out of Jail Free Card

We have never lived through a better era to be a criminal, provided your political fealty is directed toward the right person. If you are an executive facing fraud charges or a perpetrator of violent offenses, the standard calculations of the penal code may no longer apply as long as you support Donald Trump. If you’re Team Trump, the machinery of the state will actively dismantle itself to protect you. If not, good luck to you.

The Trump regime’s message is now unmistakable: rules do not apply to MAGA. Consider the recent saga of the U.S. Army pilots who took two AH-64 Apache attack helicopters on an unauthorized detour to perform a low-altitude flyby of washed-up rocker and MAGA ally Kid Rock’s Nashville home. As a former military helicopter pilot and aircraft commander, let me be clear: this is exactly the kind of stunt we are taught never to do. If I had pulled something like that, there would have been legitimate grounds to take my wings away. Instead, when the Army suspended the crew pending a standard safety and regulatory review, as is the proper procedure, Defense Secretary Pete Hegseth intervened personally, bypassing standard military discipline to announce on X: “Thank you @KidRock. @USArmy pilots suspension LIFTED. No punishment. No investigation. Carry on, patriots.” Their rule breaking was catalogued as patriotic.

Keep ReadingShow less