Collective Intelligence Increases Diagnostic Accuracy in a General Practice Setting
Matthew D. Blanchard, Stefan M. Herzog, Juliane E. Kämmer, Nikolas Zöller, Olga Kostopoulou, Ralf H. J. M. Kurvers- Health Policy
Background
General practitioners (GPs) work in an ill-defined environment where diagnostic errors are prevalent. Previous research indicates that aggregating independent diagnoses can improve diagnostic accuracy in a range of settings. We examined whether aggregating independent diagnoses can also improve diagnostic accuracy for GP decision making. In addition, we investigated the potential benefit of such an approach in combination with a decision support system (DSS).
Methods
We simulated virtual groups using data sets from 2 previously published studies. In study 1, 260 GPs independently diagnosed 9 patient cases in a vignette-based study. In study 2, 30 GPs independently diagnosed 12 patient actors in a patient-facing study. In both data sets, GPs provided diagnoses in a control condition and/or DSS condition(s). Each GP’s diagnosis, confidence rating, and years of experience were entered into a computer simulation. Virtual groups of varying sizes (range: 3–9) were created, and different collective intelligence rules (plurality, confidence, and seniority) were applied to determine each group’s final diagnosis. Diagnostic accuracy was used as the performance measure.
Results
Aggregating independent diagnoses by weighing them equally (i.e., the plurality rule) substantially outperformed average individual accuracy, and this effect increased with increasing group size. Selecting diagnoses based on confidence only led to marginal improvements, while selecting based on seniority reduced accuracy. Combining the plurality rule with a DSS further boosted performance.
Discussion
Combining independent diagnoses may substantially improve a GP’s diagnostic accuracy and subsequent patient outcomes. This approach did, however, not improve accuracy in all patient cases. Therefore, future work should focus on uncovering the conditions under which collective intelligence is most beneficial in general practice.
Highlights
We examined whether aggregating independent diagnoses of GPs can improve diagnostic accuracy. Using data sets of 2 previously published studies, we composed virtual groups of GPs and combined their independent diagnoses using 3 collective intelligence rules (plurality, confidence, and seniority). Aggregating independent diagnoses by weighing them equally substantially outperformed average individual GP accuracy, and this effect increased with increasing group size. Combining independent diagnoses may substantially improve GP’s diagnostic accuracy and subsequent patient outcomes.