Using individual patient data to adjust for indirectness did not successfully remove the bias in this case of comparative test accuracy

Junfeng Wang, Patrick Bossuyt, Ronald Geskus, Aeilko Zwinderman, Madeleine Dolleman, Simone Broer, Frank Broekmans, Ben Willem Mol, Mariska Leeflang

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objectives In comparative systematic reviews of diagnostic accuracy, inconsistencies between direct and indirect comparisons may lead to bias. We investigated whether using individual patient data (IPD) can adjust for this form of bias. Study Design and Setting We included IPD of 3 ovarian reserve tests from 32 studies. Inconsistency was defined as a statistically significant difference in relative accuracy or different comparative results between the direct and indirect evidence. We adjusted for the effect of threshold and reference standard, as well as for patient-specific variables. Results Anti-Müllerian hormone (AMH) and follicle stimulation hormone (FSH) differed significantly in sensitivity (-0.1563, P = 0.04). AMH and antral follicle count (AFC) differed significantly in sensitivity (0.1465, P < 0.01). AMH and AFC differed significantly in specificity (-0.0607, P = 0.02). The area under the curve (AUC) differed significantly between AFC and FSH (0.0948, P < 0.01) in the direct comparison but not (0.0678, P = 0.09) in the indirect comparison. The AUCs of AFC and AMH differed significantly (-0.0830, P < 0.01) in the indirect comparison but not (-0.0176, P = 0.29) in the direct comparison. These differences remained after adjusting for indirectness. Conclusion Estimates of comparative accuracy obtained through indirect comparisons are not always consistent with those obtained through direct comparisons. Using IPD to adjust for indirectness did not successfully remove the bias in this case study.

LanguageEnglish
Pages290-298
Number of pages9
JournalJournal of Clinical Epidemiology
Volume68
Issue number3
DOIs
Publication statusPublished - 1 Mar 2015

Keywords

  • Comparative meta-analysis
  • Diagnostic test accuracy
  • Generalized estimating equation
  • Individual patient data
  • Receiver operating characteristic
  • Sensitivity and specificity

ASJC Scopus subject areas

  • Epidemiology

Cite this

Wang, Junfeng ; Bossuyt, Patrick ; Geskus, Ronald ; Zwinderman, Aeilko ; Dolleman, Madeleine ; Broer, Simone ; Broekmans, Frank ; Mol, Ben Willem ; Leeflang, Mariska. / Using individual patient data to adjust for indirectness did not successfully remove the bias in this case of comparative test accuracy. In: Journal of Clinical Epidemiology. 2015 ; Vol. 68, No. 3. pp. 290-298.
@article{101333d65d6041c99cd55e47c334ff84,
title = "Using individual patient data to adjust for indirectness did not successfully remove the bias in this case of comparative test accuracy",
abstract = "Objectives In comparative systematic reviews of diagnostic accuracy, inconsistencies between direct and indirect comparisons may lead to bias. We investigated whether using individual patient data (IPD) can adjust for this form of bias. Study Design and Setting We included IPD of 3 ovarian reserve tests from 32 studies. Inconsistency was defined as a statistically significant difference in relative accuracy or different comparative results between the direct and indirect evidence. We adjusted for the effect of threshold and reference standard, as well as for patient-specific variables. Results Anti-M{\"u}llerian hormone (AMH) and follicle stimulation hormone (FSH) differed significantly in sensitivity (-0.1563, P = 0.04). AMH and antral follicle count (AFC) differed significantly in sensitivity (0.1465, P < 0.01). AMH and AFC differed significantly in specificity (-0.0607, P = 0.02). The area under the curve (AUC) differed significantly between AFC and FSH (0.0948, P < 0.01) in the direct comparison but not (0.0678, P = 0.09) in the indirect comparison. The AUCs of AFC and AMH differed significantly (-0.0830, P < 0.01) in the indirect comparison but not (-0.0176, P = 0.29) in the direct comparison. These differences remained after adjusting for indirectness. Conclusion Estimates of comparative accuracy obtained through indirect comparisons are not always consistent with those obtained through direct comparisons. Using IPD to adjust for indirectness did not successfully remove the bias in this case study.",
keywords = "Comparative meta-analysis, Diagnostic test accuracy, Generalized estimating equation, Individual patient data, Receiver operating characteristic, Sensitivity and specificity",
author = "Junfeng Wang and Patrick Bossuyt and Ronald Geskus and Aeilko Zwinderman and Madeleine Dolleman and Simone Broer and Frank Broekmans and Mol, {Ben Willem} and Mariska Leeflang",
year = "2015",
month = "3",
day = "1",
doi = "10.1016/j.jclinepi.2014.10.005",
language = "English",
volume = "68",
pages = "290--298",
journal = "Journal of Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "3",

}

Wang, J, Bossuyt, P, Geskus, R, Zwinderman, A, Dolleman, M, Broer, S, Broekmans, F, Mol, BW & Leeflang, M 2015, 'Using individual patient data to adjust for indirectness did not successfully remove the bias in this case of comparative test accuracy', Journal of Clinical Epidemiology, vol. 68, no. 3, pp. 290-298. https://doi.org/10.1016/j.jclinepi.2014.10.005

Using individual patient data to adjust for indirectness did not successfully remove the bias in this case of comparative test accuracy. / Wang, Junfeng; Bossuyt, Patrick; Geskus, Ronald; Zwinderman, Aeilko; Dolleman, Madeleine; Broer, Simone; Broekmans, Frank; Mol, Ben Willem; Leeflang, Mariska.

In: Journal of Clinical Epidemiology, Vol. 68, No. 3, 01.03.2015, p. 290-298.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Using individual patient data to adjust for indirectness did not successfully remove the bias in this case of comparative test accuracy

AU - Wang, Junfeng

AU - Bossuyt, Patrick

AU - Geskus, Ronald

AU - Zwinderman, Aeilko

AU - Dolleman, Madeleine

AU - Broer, Simone

AU - Broekmans, Frank

AU - Mol, Ben Willem

AU - Leeflang, Mariska

PY - 2015/3/1

Y1 - 2015/3/1

N2 - Objectives In comparative systematic reviews of diagnostic accuracy, inconsistencies between direct and indirect comparisons may lead to bias. We investigated whether using individual patient data (IPD) can adjust for this form of bias. Study Design and Setting We included IPD of 3 ovarian reserve tests from 32 studies. Inconsistency was defined as a statistically significant difference in relative accuracy or different comparative results between the direct and indirect evidence. We adjusted for the effect of threshold and reference standard, as well as for patient-specific variables. Results Anti-Müllerian hormone (AMH) and follicle stimulation hormone (FSH) differed significantly in sensitivity (-0.1563, P = 0.04). AMH and antral follicle count (AFC) differed significantly in sensitivity (0.1465, P < 0.01). AMH and AFC differed significantly in specificity (-0.0607, P = 0.02). The area under the curve (AUC) differed significantly between AFC and FSH (0.0948, P < 0.01) in the direct comparison but not (0.0678, P = 0.09) in the indirect comparison. The AUCs of AFC and AMH differed significantly (-0.0830, P < 0.01) in the indirect comparison but not (-0.0176, P = 0.29) in the direct comparison. These differences remained after adjusting for indirectness. Conclusion Estimates of comparative accuracy obtained through indirect comparisons are not always consistent with those obtained through direct comparisons. Using IPD to adjust for indirectness did not successfully remove the bias in this case study.

AB - Objectives In comparative systematic reviews of diagnostic accuracy, inconsistencies between direct and indirect comparisons may lead to bias. We investigated whether using individual patient data (IPD) can adjust for this form of bias. Study Design and Setting We included IPD of 3 ovarian reserve tests from 32 studies. Inconsistency was defined as a statistically significant difference in relative accuracy or different comparative results between the direct and indirect evidence. We adjusted for the effect of threshold and reference standard, as well as for patient-specific variables. Results Anti-Müllerian hormone (AMH) and follicle stimulation hormone (FSH) differed significantly in sensitivity (-0.1563, P = 0.04). AMH and antral follicle count (AFC) differed significantly in sensitivity (0.1465, P < 0.01). AMH and AFC differed significantly in specificity (-0.0607, P = 0.02). The area under the curve (AUC) differed significantly between AFC and FSH (0.0948, P < 0.01) in the direct comparison but not (0.0678, P = 0.09) in the indirect comparison. The AUCs of AFC and AMH differed significantly (-0.0830, P < 0.01) in the indirect comparison but not (-0.0176, P = 0.29) in the direct comparison. These differences remained after adjusting for indirectness. Conclusion Estimates of comparative accuracy obtained through indirect comparisons are not always consistent with those obtained through direct comparisons. Using IPD to adjust for indirectness did not successfully remove the bias in this case study.

KW - Comparative meta-analysis

KW - Diagnostic test accuracy

KW - Generalized estimating equation

KW - Individual patient data

KW - Receiver operating characteristic

KW - Sensitivity and specificity

UR - http://www.scopus.com/inward/record.url?scp=84923443693&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2014.10.005

DO - 10.1016/j.jclinepi.2014.10.005

M3 - Article

VL - 68

SP - 290

EP - 298

JO - Journal of Epidemiology

T2 - Journal of Epidemiology

JF - Journal of Epidemiology

SN - 0895-4356

IS - 3

ER -