Are artificial intelligence based chatbots reliable sources for patients regarding orthodontics?
Tuğba Haliloğlu Özkan, Ahmet Hüseyin Acar, Enes Özkan, Mustafa Düzyol, Elif Aybüke ÖztürkObjectives:
The objective of this study was to conduct a comprehensive and patient-centered evaluation of chatbot responses within the field of orthodontics, comparing three prominent chatbot platforms: ChatGPT-4, Microsoft Copilot, and Google Gemini.
Material and Methods:
Twenty orthodontic-related queries were presented to ChatGPT-4, Microsoft Copilot, and Google Gemini by ten orthodontic experts. To assess the accuracy and completeness of responses, a Likert scale (LS) was employed, while the clarity of responses was evaluated using a Global Quality Scale (GQS). Statistical analyses included One-way analysis of variance and post-hoc Tukey tests to assess the data, and a Pearson correlation test was used to determine the relationship between variables.
Results:
The results indicated that ChatGPT-4 (1.69 ± 0.10) and Microsoft Copilot (1.68 ± 0.10) achieved significantly higher LS scores compared to Google Gemini (2.27 ± 0.53) (P < 0.05). However, the GQS scores, which were 4.01 ± 0.31 for ChatGPT-4, 3.92 ± 0.60 for Google Gemini, and 4.09 ± 0.15 for Microsoft Copilot, showed no significant differences among the three chatbots (P > 0.05).
Conclusion:
While these chatbots generally handle basic orthodontic queries well, they show significant differences in responses to complex scenarios. ChatGPT-4 and Microsoft Copilot outperform Google Gemini in accurately addressing scenario-based questions, highlighting the importance of strong language comprehension, knowledge access, and advanced algorithms. This underscores the need for continued improvements in chatbot technology.