Effects of double data extraction on errors in evidence synthesis: a crossover, multicenter, investigator-blinded, randomized controlled trial
Lijun Tang, Ruoxi Wang, Suhail A R Doi, Luis Furuya-Kanamori, Lifeng Lin, Zongshi Qin, Fangbiao Tao, Chang XuAbstract
Objectives
The objective was to investigate the role of double extraction in reducing data errors in evidence synthesis for pharmaceutical and non-pharmaceutical interventions.
Design
Crossover randomized controlled trial (RCT).
Setting
University and hospital with teaching programs in evidence-based medicine.
Participants
One hundred eligible participants were randomly assigned in a 1:1 ratio to perform data extraction tasks for either 10 RCTs of pharmaceutical interventions or 10 RCTs of non-pharmaceutical interventions, followed by a crossover pattern and a further cross-checking process (double extraction). Only data on binary adverse outcomes were extracted.
Intervention
Double data extraction versus single extraction.
Primary and secondary outcome measures
The primary outcome was the error rate before and after the cross-checking process. The secondary outcome was the absolute difference in error rates. Error rates were assessed at both the study level and the cell level.
Results
Error rates in the pharmaceutical and non-pharmaceutical groups were 64.65% and 59.90%, respectively, with an absolute difference of 4.75% and an odds ratio (OR) of 1.29 [95% confidence interval (CI): 1.06–1.57, P = .01] when measured at the study level. After cross-checking, error rates decreased to 44.88% and 39.54%, with the difference between the two groups remaining at 5.34%, and an OR of 1.27 (95%CI: 1.1–1.46; P < .01). Similar differences were observed when measured at the cell level.
Conclusion
Although double extraction reduced data errors, the error rate remained high after the process. Evidence synthesis research may consider triple data extraction to further minimize potential data errors.
Trial registration number
Chinese Clinical Trial Registry Center (Identifier: ChiCTR2200062206).