Suicide is a leading cause of death in the United States, representing a critical public health challenge. Predictive modeling can help identify individuals at high-risk, but biases may compromise their effectiveness. In this study, we perform innovative data integration and advanced model evaluation to quantify bias in suicide risk classification, while also exploring strategies to mitigate it, such as feature-specific model analysis. We examine three model specifications for predicting suicide methods (firearm, hanging & drowning, poisoning, and other) using the following approaches: (1) patient-level characteristics, (2) a combination of patient characteristics and high-level social determinants of health (SDoH) variables, and (3) patient characteristics integrated with more detailed SDoH topic variables. Model performance is assessed using various metrics. XGBoost consistently outperforms the other models, making it the preferred approach for this task. To assess algorithmic bias, we introduce a disparity metric based on the F1 score to identify disparities both across overall subgroups and within the top 5% of high-risk strata, which is essential for implementing effective preventive interventions. Our analysis uncovers significant disparities linked to individual-level characteristics, regardless of the inclusion of ecological SDoH factors. Whether examining the entire dataset or focusing specifically on high-risk groups, we find that using separate models for distinct subgroups yields higher disparity scores compared to a single, unified model. These findings underscore the importance of addressing algorithmic bias to ensure that suicide risk prediction models do not perpetuate existing disparities and remain equitable across all subgroups.
