Skip to content

Latest Stories

Top Stories

AI could help remove bias from medical research and data

Researcher looks at mammography test

Artificial intelligence can help root out racial bias in health care, but only if the programmers can create the software so it doesn't make the same mistakes people make, like misreading mammograms results, writes Pearl.

Anne-Christine Poujoulat/AFP via Getty Images
Pearl is a clinical professor of plastic surgery at the Stanford University School of Medicine and is on the faculty of the Stanford Graduate School of Business. He is a former CEO of The Permanente Medical Group.

This is the second entry in a two-part op-ed series on institutional racism in American medicine.

A little over a year before the coronavirus pandemic reached our shores, the racism problem in U.S. health care was making big headlines.

But it wasn't doctors or nurses being accused of bias. Rather, a study published in Science concluded that a predictive health care algorithm had, itself, discriminated against Black patients.

The story originated with Optum, a subsidiary of insurance giant UnitedHealth Group, which had designed an application to identify high-risk patients with untreated chronic diseases. The company's ultimate goal was to help re-distribute medical resources to those who'd benefit most from added care. And to figure out who was most in need, Optum's algorithm assessed the cost of each patient's past treatments.

Unaccounted for in the algorithm's design was this essential fact: The average Black patient receives $1,800 less per year in total medical care than a white person with the same set of health problems. And, sure enough, when the researchers went back and re-ranked patients by their illnesses (rather than the cost of their care), the percentage of Black patients who should have been enrolled in specialized care programs jumped from 18 percent to 47 percent.

Sign up for The Fulcrum newsletter

Journalists and commentators pinned the blame for racial bias on Optum's algorithm. In reality, technology wasn't the problem. At issue were the doctors who failed to provide sufficient medical care to the Black patients in the first place. Meaning, the data was faulty because humans failed to provide equitable care.

Artificial intelligence and algorithmic approaches can only be as accurate, reliable and helpful as the data they're given. If the human inputs are unreliable, the data will be, as well.

Let's use the identification of breast cancer as an example. As much as one-third of the time, two radiologists looking at the same mammogram will disagree on the diagnosis. Therefore, if AI software were programmed to act like humans, the technology would be wrong one-third of the time.

Instead, AI can store and compare tens of thousands of mammogram images — comparing examples of women with cancer and without — to detect hundreds of subtle differences that humans often overlook. It can remember all those tiny differences when reviewing new mammograms, which is why AI is already estimated to be 10 percent more accurate than the average radiologist.

What AI can't recognize is whether it's being fed biased or incorrect information. Adjusting for bias in research and data aggregation requires that humans acknowledge their faulty assumptions and decisions, and then modify the inputs accordingly.

Correcting these types of errors should be standard practice by now. After all, any research project that seeks funding and publication is required to include an analysis of potential bias, based on the study's participants. As an example, investigators who want to compare people's health in two cities would be required to modify the study's design if they failed to account for major differences in age, education or other factors that might inappropriately tilt the results.

Given how often data is flawed, the possibility of racial bias should be explicitly factored into every AI project. With universities and funding agencies increasingly focused on racial issues in medicine, this expectation has the potential to become routine in the future. Once it is, AI will force researchers to confront bias in health care. As a result, the conclusions and recommendations they provide will be more accurate and equitable.

Thirteen months into the pandemic, Covid-19 continues to kill Black individuals at a rate three times higher than white people. For years, health plans and hospital leaders have talked about the need to address health disparities like these. And yet, despite good intentions, the solutions they put forth always look a lot like the failed efforts of the past.

Addressing systemic racism in medicine requires that we analyze far more data (all at once) than we do today. AI is the perfect application for this task. What we need is a national commitment to use these types of technologies to answer medicine's most urgent questions.

There is no antidote to the problem of racism in medicine. But combining AI with a national commitment to root out bias in health care would be a good start, putting our medical system on a path toward antiracism.

Read More

Rep. Gus Bilirakis and Rep. Ayanna Pressley

Rep. Gus Bilirakis and Rep. Ayanna Pressley won the Congressional Management Foundation's Democracy Award for Constituent Accountability and Accessibility.

Official portraits

Some leaders don’t want to be held accountable. These two expect it.

Fitch is president and CEO of the Congressional Management Foundation and a former congressional staffer.

There is probably no more important concept in the compact between elected officials and those who elect them than accountability. One of the founding principles of American democracy is that members of Congress are ultimately accountable to their constituents, both politically and morally. Most members of Congress get this, but how they demonstrate and implement that concept varies. The two winners of the Congressional Management Foundation’s Democracy Award for Constituent Accountability and Accessibility clearly understand and excel at this concept.

Keep ReadingShow less
Woman speaking at a microphone

Rep. Lucy McBath is the first lawmaker from Georgia to win a Democracy Awarrd.

Michael Brochstein/SOPA Images/LightRocket via Getty Images

Surprise: Some great public servants are actually members of Congress

Fitch is the president and CEO of the Congressional Management Foundation and a former congressional staffer.

TheCongressional Management Foundation today announced the winners of the seventh annual Democracy Awards, CMF’s program recognizing non-legislative achievement and performance in congressional offices and by members of Congress. Two members of Congress, one Democrat and one Republican, are recognized in four categories related to their work in Congress.

Americans usually only hear about Congress when something goes wrong. The Democracy Awards shines a light on Congress when it does something right. These members of Congress and their staff deserve recognition for their work to improve accountability in government, modernize their work environments and serve their constituents.

Keep ReadingShow less

Can George Washington inspire Biden to greatness?

Clancy is co-founder of Citizen Connect and board member of the Bridge Alliance Education Fund. Citizen Connect is an initiative of the Bridge Alliance Education Fund, which also operates The Fulcrum.

King George III reputedly said George Washington was the greatest man in the world for voluntarily relinquishing power. The indisputable fact is that Washington’s action remains remarkable in human history. And he actually did it at least two times.

On Dec. 23, 1783, Washington resigned his commission as commander of the Continental Army and returned to Mount Vernon. He did it again when he declined to run for a third term as president by publishing his Farewell Address on Sept. 19, 1796. In June 1799 Washington was yet again urged to run for president and declined.

His reasoning on each occasion was a complex mix of the personal and political, but the bedrock was an unwavering commitment to put the good of the nation above personal gain and the factions that would ultimately become our toxic party system.

Keep ReadingShow less
Joe Biden at the debate

After his disastrous peformance at the debate, President Biden needs to exit the race, writes Breslin.

Kyle Mazza/Anadolu via Getty Images

Getting into the highest offices is hard. Getting out is harder.

Breslin is the Joseph C. Palamountain Jr. Chair of Political Science at Skidmore College and author of “A Constitution for the Living: Imagining How Five Generations of Americans Would Rewrite the Nation’s Fundamental Law.”

This is the latest in “A Republic, if we can keep it,” a series to assist American citizens on the bumpy road ahead this election year. By highlighting components, principles and stories of the Constitution, Breslin hopes to remind us that the American political experiment remains, in the words of Alexander Hamilton, the “most interesting in the world.”

Getting into America’s highest political offices is hard. Getting out is harder.

President Joe Biden’s disastrous debate performance has intensified calls for him to step aside. Not even 24 hours after his poor showing, The New York Times took the extraordinary and unprecedented position that the sitting president should immediately pass the torch to a more energetic and electable candidate. “The greatest public service Mr. Biden can now perform,” the editorial board declared, “is to announce that he will not continue to run for re-election.”

Keep ReadingShow less
Donald Trump on stage at the debate

Donald Trump's emphasis on the power of negative information gave him an advantage at Thursday's debate.

Kyle Mazza/Anadolu via Getty Images

How harnessing the power of bad helped Trump win the debate

Assari is an associate professor of public health and Internal Medicine at Charles R. Drew University of Medicine and Science. He is a member of the Scholars Strategy Network.

On Thursday, we witnessed a debate between President Joe Biden and former President Donald Trump. What captured most of the audience's attention were Biden's occasional stumbles and Trump's numerous statements inaccuracies. Biden appeared humble and policy-focused, while Trump was loud, assertive and well-spoken.

However, another significant aspect was Trump's rhetoric, which was often filled with threats, fear and loss, especially on topics like immigration and crime. In contrast, Biden's points were primarily centered around policies and statistics aimed at benefiting Americans overall, with a particular focus on the vulnerable.

Keep ReadingShow less