Linking quality indicators to clinical trials: An automated approach

Enrico Coiera, Miew Keen Choong, Guy Tsafnat, Peter Hibbert, William B. Runciman

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Objective: Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. Design: A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. Intervention: Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. Main Outcome Measure: Number of care indicators that mapped to outcome measures in clinical trials. Results: While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method. Conclusions: We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

LanguageEnglish
Pages571-578
Number of pages8
JournalInternational Journal for Quality in Health Care
Volume29
Issue number4
DOIs
Publication statusPublished - 1 Jan 2017

Keywords

  • Clinical trials
  • Concept mapping
  • Indicator
  • Quality of health care
  • Text mining

ASJC Scopus subject areas

  • Health Policy
  • Public Health, Environmental and Occupational Health

Cite this

Coiera, Enrico ; Choong, Miew Keen ; Tsafnat, Guy ; Hibbert, Peter ; Runciman, William B. / Linking quality indicators to clinical trials : An automated approach. In: International Journal for Quality in Health Care. 2017 ; Vol. 29, No. 4. pp. 571-578.
@article{89b492b43e5e466bbdc14a7071506e56,
title = "Linking quality indicators to clinical trials: An automated approach",
abstract = "Objective: Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. Design: A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. Intervention: Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. Main Outcome Measure: Number of care indicators that mapped to outcome measures in clinical trials. Results: While only 13{\%} of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68{\%}) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71{\%}) Level I and II evidence-based indicators, and 268 of 370 (72{\%}) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7{\%} (4 of 70) were mentioned in the trial reports but were missed by our method. Conclusions: We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.",
keywords = "Clinical trials, Concept mapping, Indicator, Quality of health care, Text mining",
author = "Enrico Coiera and Choong, {Miew Keen} and Guy Tsafnat and Peter Hibbert and Runciman, {William B.}",
year = "2017",
month = "1",
day = "1",
doi = "10.1093/intqhc/mzx076",
language = "English",
volume = "29",
pages = "571--578",
journal = "International Journal for Quality in Health Care",
issn = "1353-4505",
publisher = "Oxford University Press",
number = "4",

}

Linking quality indicators to clinical trials : An automated approach. / Coiera, Enrico; Choong, Miew Keen; Tsafnat, Guy; Hibbert, Peter; Runciman, William B.

In: International Journal for Quality in Health Care, Vol. 29, No. 4, 01.01.2017, p. 571-578.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Linking quality indicators to clinical trials

T2 - International Journal for Quality in Health Care

AU - Coiera, Enrico

AU - Choong, Miew Keen

AU - Tsafnat, Guy

AU - Hibbert, Peter

AU - Runciman, William B.

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Objective: Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. Design: A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. Intervention: Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. Main Outcome Measure: Number of care indicators that mapped to outcome measures in clinical trials. Results: While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method. Conclusions: We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

AB - Objective: Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. Design: A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. Intervention: Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. Main Outcome Measure: Number of care indicators that mapped to outcome measures in clinical trials. Results: While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method. Conclusions: We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

KW - Clinical trials

KW - Concept mapping

KW - Indicator

KW - Quality of health care

KW - Text mining

UR - http://www.scopus.com/inward/record.url?scp=85030456951&partnerID=8YFLogxK

U2 - 10.1093/intqhc/mzx076

DO - 10.1093/intqhc/mzx076

M3 - Article

VL - 29

SP - 571

EP - 578

JO - International Journal for Quality in Health Care

JF - International Journal for Quality in Health Care

SN - 1353-4505

IS - 4

ER -