Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems

Robert Jorissen, Matthew Croxford, Ian T. Jones, Robyn L. Ward, Nicholas J. Hawkins, Peter Gibbs, Oliver M. Sieber

Research output: Contribution to journalArticle

Abstract

Adjuvant! Online Inc (A!O), the Memorial Sloan Kettering Cancer Center (MSKCC), MD Anderson (MDA) and Mayo Clinic (MC) provide calculators to predict survival probabilities for patients with resected early-stage colon cancer, trained on data from United States (US) patient cohorts or patients enrolled in international clinical trials. Limited data exist on the transferability of calculators across healthcare systems. Calculator transferability to Australian community practice was evaluated for 1,401 stage II/III patients. Calibration and discrimination were assessed for overall (OS), cancer-specific (CSS) or recurrence-free survival (RFS). The US patient cohort-based calculators, A!O, MSKCC and MDA, significantly overestimated risks of recurrence and death in Australian patients, with 5-year OS, CSS and RFS prediction differences of −6.5% to −9.9%, −9.1% to −14.4% and − 3.8% to −6.8%, respectively (p < 0.001). Significant heterogeneity in calibration was observed for subgroups by tumor stage and treatment, age, gender, tumor location, ECOG and ASA score. Calibration appeared acceptable for the clinical trial patient-based MC calculator, but restricted tool applicability (stage III patients, ≥12 examined lymph nodes, receiving adjuvant treatment) limited the sample size. Compared to AJCC 7th edition tumor staging, calculators showed improved discrimination for OS, but no improvement for CSS and RFS. In conclusion, deficiencies in calibration limited transferability of US patient cohort-based survival calculators for early-stage colon cancer to the setting of Australian community practice. Our results demonstrate the utility for multi-feature survival calculators to improve OS predictions but highlight the importance for performance assessment of tools prior to implementation in an external health care setting.

LanguageEnglish
Pages132-142
Number of pages11
JournalInternational Journal of Cancer
Volume145
Issue number1
DOIs
Publication statusPublished - 1 Jul 2019

Keywords

  • calibration
  • colon cancer
  • discrimination
  • survival calculator

ASJC Scopus subject areas

  • Oncology
  • Cancer Research

Cite this

Jorissen, Robert ; Croxford, Matthew ; Jones, Ian T. ; Ward, Robyn L. ; Hawkins, Nicholas J. ; Gibbs, Peter ; Sieber, Oliver M. / Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems. In: International Journal of Cancer. 2019 ; Vol. 145, No. 1. pp. 132-142.
@article{23741e43f49d434183b3990738f4c303,
title = "Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems",
abstract = "Adjuvant! Online Inc (A!O), the Memorial Sloan Kettering Cancer Center (MSKCC), MD Anderson (MDA) and Mayo Clinic (MC) provide calculators to predict survival probabilities for patients with resected early-stage colon cancer, trained on data from United States (US) patient cohorts or patients enrolled in international clinical trials. Limited data exist on the transferability of calculators across healthcare systems. Calculator transferability to Australian community practice was evaluated for 1,401 stage II/III patients. Calibration and discrimination were assessed for overall (OS), cancer-specific (CSS) or recurrence-free survival (RFS). The US patient cohort-based calculators, A!O, MSKCC and MDA, significantly overestimated risks of recurrence and death in Australian patients, with 5-year OS, CSS and RFS prediction differences of −6.5{\%} to −9.9{\%}, −9.1{\%} to −14.4{\%} and − 3.8{\%} to −6.8{\%}, respectively (p < 0.001). Significant heterogeneity in calibration was observed for subgroups by tumor stage and treatment, age, gender, tumor location, ECOG and ASA score. Calibration appeared acceptable for the clinical trial patient-based MC calculator, but restricted tool applicability (stage III patients, ≥12 examined lymph nodes, receiving adjuvant treatment) limited the sample size. Compared to AJCC 7th edition tumor staging, calculators showed improved discrimination for OS, but no improvement for CSS and RFS. In conclusion, deficiencies in calibration limited transferability of US patient cohort-based survival calculators for early-stage colon cancer to the setting of Australian community practice. Our results demonstrate the utility for multi-feature survival calculators to improve OS predictions but highlight the importance for performance assessment of tools prior to implementation in an external health care setting.",
keywords = "calibration, colon cancer, discrimination, survival calculator",
author = "Robert Jorissen and Matthew Croxford and Jones, {Ian T.} and Ward, {Robyn L.} and Hawkins, {Nicholas J.} and Peter Gibbs and Sieber, {Oliver M.}",
year = "2019",
month = "7",
day = "1",
doi = "10.1002/ijc.32100",
language = "English",
volume = "145",
pages = "132--142",
journal = "International Journal of Cancer",
issn = "0020-7136",
publisher = "Wiley-Liss Inc.",
number = "1",

}

Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems. / Jorissen, Robert; Croxford, Matthew; Jones, Ian T.; Ward, Robyn L.; Hawkins, Nicholas J.; Gibbs, Peter; Sieber, Oliver M.

In: International Journal of Cancer, Vol. 145, No. 1, 01.07.2019, p. 132-142.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems

AU - Jorissen, Robert

AU - Croxford, Matthew

AU - Jones, Ian T.

AU - Ward, Robyn L.

AU - Hawkins, Nicholas J.

AU - Gibbs, Peter

AU - Sieber, Oliver M.

PY - 2019/7/1

Y1 - 2019/7/1

N2 - Adjuvant! Online Inc (A!O), the Memorial Sloan Kettering Cancer Center (MSKCC), MD Anderson (MDA) and Mayo Clinic (MC) provide calculators to predict survival probabilities for patients with resected early-stage colon cancer, trained on data from United States (US) patient cohorts or patients enrolled in international clinical trials. Limited data exist on the transferability of calculators across healthcare systems. Calculator transferability to Australian community practice was evaluated for 1,401 stage II/III patients. Calibration and discrimination were assessed for overall (OS), cancer-specific (CSS) or recurrence-free survival (RFS). The US patient cohort-based calculators, A!O, MSKCC and MDA, significantly overestimated risks of recurrence and death in Australian patients, with 5-year OS, CSS and RFS prediction differences of −6.5% to −9.9%, −9.1% to −14.4% and − 3.8% to −6.8%, respectively (p < 0.001). Significant heterogeneity in calibration was observed for subgroups by tumor stage and treatment, age, gender, tumor location, ECOG and ASA score. Calibration appeared acceptable for the clinical trial patient-based MC calculator, but restricted tool applicability (stage III patients, ≥12 examined lymph nodes, receiving adjuvant treatment) limited the sample size. Compared to AJCC 7th edition tumor staging, calculators showed improved discrimination for OS, but no improvement for CSS and RFS. In conclusion, deficiencies in calibration limited transferability of US patient cohort-based survival calculators for early-stage colon cancer to the setting of Australian community practice. Our results demonstrate the utility for multi-feature survival calculators to improve OS predictions but highlight the importance for performance assessment of tools prior to implementation in an external health care setting.

AB - Adjuvant! Online Inc (A!O), the Memorial Sloan Kettering Cancer Center (MSKCC), MD Anderson (MDA) and Mayo Clinic (MC) provide calculators to predict survival probabilities for patients with resected early-stage colon cancer, trained on data from United States (US) patient cohorts or patients enrolled in international clinical trials. Limited data exist on the transferability of calculators across healthcare systems. Calculator transferability to Australian community practice was evaluated for 1,401 stage II/III patients. Calibration and discrimination were assessed for overall (OS), cancer-specific (CSS) or recurrence-free survival (RFS). The US patient cohort-based calculators, A!O, MSKCC and MDA, significantly overestimated risks of recurrence and death in Australian patients, with 5-year OS, CSS and RFS prediction differences of −6.5% to −9.9%, −9.1% to −14.4% and − 3.8% to −6.8%, respectively (p < 0.001). Significant heterogeneity in calibration was observed for subgroups by tumor stage and treatment, age, gender, tumor location, ECOG and ASA score. Calibration appeared acceptable for the clinical trial patient-based MC calculator, but restricted tool applicability (stage III patients, ≥12 examined lymph nodes, receiving adjuvant treatment) limited the sample size. Compared to AJCC 7th edition tumor staging, calculators showed improved discrimination for OS, but no improvement for CSS and RFS. In conclusion, deficiencies in calibration limited transferability of US patient cohort-based survival calculators for early-stage colon cancer to the setting of Australian community practice. Our results demonstrate the utility for multi-feature survival calculators to improve OS predictions but highlight the importance for performance assessment of tools prior to implementation in an external health care setting.

KW - calibration

KW - colon cancer

KW - discrimination

KW - survival calculator

UR - http://www.scopus.com/inward/record.url?scp=85060327901&partnerID=8YFLogxK

U2 - 10.1002/ijc.32100

DO - 10.1002/ijc.32100

M3 - Article

VL - 145

SP - 132

EP - 142

JO - International Journal of Cancer

T2 - International Journal of Cancer

JF - International Journal of Cancer

SN - 0020-7136

IS - 1

ER -