3 ways Google is integrating healthcare equity into AI
3 ways Google is integrating healthcare equity into AI

The goal of health equity is to ensure that everyone has a fair and equitable opportunity to achieve their highest level of health. The reality is the opposite for too many people—including people of color, women, those in rural communities, and other historically marginalized populations. As Google’s Chief Health Investment Officer, my team is committed to ensuring we build AI-based health tools responsibly and fairly.

At our annual health event, The Check Up, we revealed three ways we are helping to achieve a fairer future.

Our recent research on identifying and mitigating bias

As medical AI rapidly advances, it is critical that we develop tools and resources that can be used to identify and mitigate biases that could negatively impact health outcomes. Our new research paper, “A Toolkit for Revealing Health Equity Harms and Biases in Large Language Models,” is a step in that direction. This paper provides a framework for how to assess whether medical large language models (LLMs) may perpetuate historical biases and provides a collection of seven competitive testing datasets called “EquityMedQA” as a benchmark.

These tools are based on literature on health inequities, actual model failures, and participatory input from equity experts. We used these tools to evaluate our own large language models, and they are now available to the research community and beyond.

A new framework for measuring health equity within AI models

A group of health equity researchers, social scientists, clinicians, bioethicists, statisticians, and AI researchers came together at Google to develop a framework for building AI that avoids creating and reinforcing unfair biases.

This framework, which is called HEAL (Health Equity Assessment of Machine Learning Performance), is designed to assess the likelihood that AI technology will work equitably and prevent the deployment of AI models that could exacerbate disparities – especially for groups with poorer health average scores. The four-step process includes:

  1. Identifying factors associated with health inequities and defining AI performance indicators.
  2. Identifying and quantifying existing disparities in health outcomes.
  3. Measuring the performance of the AI ​​tool for each subpopulation.
  4. Assessing the Likelihood of an Artificial Intelligence Tool to Prioritize Performance on Health Disparities.

We have already used this framework to test a dermatology AI model. The results showed that although this model performs equally well across racial, ethnic, and gender subgroups, there are improvements we can make to perform better for older age groups. The framework found that when it came to assessing cancerous conditions, such as melanoma, the model performed equally across age groups, but for non-cancerous conditions, such as eczema, it did not perform as well in the 70 and over age group.

We will continue to apply the framework to healthcare AI models in the future and develop and refine the framework in the process.

A more representative data set for advances in dermatology

Today, many dermatology datasets are not representative of the population, which limits developers from building fair AI models. Current dataset images are often captured in a clinical setting and may not reflect different body parts, different levels of severity of a condition, or different skin tones, ages, genders, and more. And they’re mostly focused on severe problems — like skin cancer — rather than more common problems like allergic, inflammatory or infectious conditions.

To create a more representative set of images, we partnered with Stanford Medicine on the Skin Condition Imaging Network (SCIN). Thousands of people contributed more than 10,000 real-world dermatology images to create this open-access dataset. Dermatologists and research teams then helped identify diagnoses on each image and labeled them based on two skin tone scales to ensure it included a broad collection of skin conditions and types.

Scientists and physicians can now use the SCIN dataset to help them develop tools to identify dermatological problems, conduct dermatology-related research, and expose healthcare students to more examples of skin conditions and their manifestations in different types skin.

We are at the beginning of this journey, but we are determined to make a difference. We believe that working with partners and sharing our learning can help build a healthier future for everyone, regardless of their background or location.

By admin

Leave a Reply

Your email address will not be published. Required fields are marked *