Abstract
Background: This work evaluated algorithmic bias on biomarkers using electronic pathology reports of female breast cancer across five subgroups: National Cancer Institute Surveillance, Epidemiology, and End Results (SEER) registry, racial categories, Hispanic ethnicity, age at diagnosis, and socioeconomic status.
Methods: The Framework for Exploring Scalable Computational Oncology and 594,875 electronic pathology reports of 178,121 tumors from the Kentucky, Louisiana, New Jersey, New Mexico, Seattle, and Utah SEER registries were used to train the Multi-Task Convolutional Neural Network algorithm to extract biomarker test results for patients with breast cancer. We evaluated model performance on 15\% of the dataset using a balanced error rate (BER): the majority class in each subgroup was the class with the most data.
Results: BER showed no significant bias in extracting biomarkers (ER, PR, HER2) toward the majority class in the racial, Hispanic ethnicity, age at diagnosis, or socioeconomic subgroups (BER ratio $<$1.25). We found differences in predictive accuracy between registries, with the highest predictive accuracy in the registry that contributed the most data (Seattle Registry, BER ratios for all registries $>$1.25).
Conclusion: Our results indicate that prediction accuracy can be affected by differences in the volume and distribution of data across registries; however, the algorithm results were not biased by race, ethnicity, age, or socioeconomic status. Training AI models with data from the SEER registries, which capture all cancer reports in a catchment area, may provide insight into how this data may bias models.
Impact: AI tools designed to expedite information extraction from clinical records could accelerate clinical trial matching and improve care. However, a thorough evaluation of algorithmic biases that may affect equality in clinical care is a critical step before deployment.
