Chat GPT beats MD response to patient questions

Communication is an essential component of the doctor-patient relationship. Clinical outcomes depend upon the quality of this interaction. Feeling understood and respected facilitates a therapeutic alliance that promotes compliance and a shared sense of purpose. However, shortened visits, overwhelming patient loads and pervasive physician burnout now make good healthcare almost impossible.

COVID accelerated the adoption of virtual health care and electronic patient messages. This has only increased the doctor’s workload and burnout. A recent study published in JAMA Internal Medicine sought to determine whether an artificial intelligence chatbot assistant could provide responses to patient questions of comparable quality and empathy to physician responses. The results were unexpected.

A database of questions from Reddit’s r/AskDocs, a public social media forum, provided 195 exchanges where a physician responded to a question. Doctor and chatbot responses were evaluated by a team of licensed healthcare professionals. Evaluators chose the “better” response on the basis of the quality of information (very poor, poor, acceptable, good, or very good) and the empathy or bedside manner of the response (not empathic, slightly empathetic, moderately empathetic, empathetic, and very empathetic).

Chatbot responses were judged superior to physician responses in 78.6% of the evaluations. A good or very good assessment was assigned to 78.5% of chatbot responses versus 22.1% of physician responses.

The results for empathy were stunning. The proportion of responses rated empathetic or very empathetic was 45.1% for the chatbot and 4.6% for physicians.

This is good and bad news. Health care professionals preferred chatbot responses to physician responses 4 to 1, a damning indictment of doctors’ performance. At the same time, this research suggests that AI messaging might save physicians time and decrease burnout while positively affecting patient care and outcomes.

The question of whether empathy can be taught has been a perennial debate in medical education. Clearly the current structure of healthcare delivery does not lend itself to empathic behavior. Perhaps machines will teach us how to be more humane.