The lung allocation score and other available models lack predictive accuracy for post-lung transplant survival

Abstract

Background: Improved predictive models are needed in lung transplantation in the setting of a proposed allocation system that incorporates longer-term post-transplant survival in the United States. Allocation systems require accurate mortality predictions to justly allocate organs.

Methods: Utilizing the United Network for Organ Sharing database (2005-2017), we fit models to predict 1-year mortality based on the Lung Allocation Score (LAS), the Chan, et al, 2019 model, a novel "clinician" model (a priori clinician selection of pre-transplant covariates), and two machine learning models (Least Absolute Shrinkage and Selection Operator; LASSO and Random Forests) for predicting 1-year and 3-year post-transplant mortality. We compared predictive accuracy among models. We evaluated the calibration of models by comparing average predicted probability vs observed outcome per decile. We repeated analyses fit for 3-year mortality, disease category, including donor covariates, and LAS era.

Results: The area under the cure for all models was low, ranging from 0.55 to 0.62. All exhibited reasonable negative predictive values (0.87-0.90), but the positive predictive value for was poor (all <0.25). Evaluating LAS calibration found 1-year post-transplant estimates consistently overestimated risk of mortality, with greater differences in higher deciles. LASSO, Random Forests, and clinician models showed no improvement when evaluated by disease category or with the addition of donor covariates and performed worse for 3-year outcomes.

Conclusions: The LAS overestimated patients' risk of post-transplant death, thus underestimating transplant benefit in the sickest candidates. Novel models based on pre-transplant recipient covariates failed to improve prediction. There should be wariness in post-transplant survival predictions from available models.

Keywords: lung allocation score; lung transplant outcomes; lung transplantation; organ allocation; predictive accuracy.