Temporal shift and predictive performance of machine learning for heart transplant outcomes

Abstract

BACKGROUND: Outcome prediction following heart transplant is critical to explaining risks and benefits to patients and decision-making when considering potential organ offers. Given the large number of potential variables to be considered, this task may be most efficiently performed using machine learning (ML). We trained and tested ML and statistical algorithms to predict outcomes following cardiac transplant using the United Network of Organ Sharing (UNOS) database.

METHODS: We included 59,590 adult and 8,349 pediatric patients enrolled in the UNOS database between January 1994 and December 2016 who underwent cardiac transplantation. We evaluated 3 classification and 3 survival methods. Algorithms were evaluated using shuffled 10-fold cross-validation (CV) and rolling CV. Predictive performance for 1 year and 90 days all-cause mortality was characterized using the area under the receiver-operating characteristic curve (AUC) with 95% confidence interval.

RESULTS: In total, 8,394 (12.4%) patients died within 1 year of transplant. For predicting 1-year survival, using the shuffled 10-fold CV, Random Forest achieved the highest AUC (0.893; 0.889-0.897) followed by XGBoost and logistic regression. In the rolling CV, prediction performance was more modest and comparable among the models with XGBoost and Logistic regression achieving the highest AUC 0.657 (0.647-0.667) and 0.641(0.631-0.651), respectively. There was a trend toward higher prediction performance in pediatric patients.

CONCLUSIONS: Our study suggests that ML and statistical models can be used to predict mortality post-transplant, but based on the results from rolling CV, the overall prediction performance will be limited by temporal shifts inpatient and donor selection. (C) 2022 International Society for Heart and Lung Transplantation. All rights reserved.