Harvard Medical School researchers are developing Dr. CaBot as a medical education tool. The system, which happens to operate across both presentation and written formats, shows how it reasons through a case, thereby offering what’s called a differential diagnosis, which is a comprehensive list of possible conditions, which goes on to explain what’s going on and also narrows down the possibilities till the time it reaches its final diagnosis.
The ability of Dr. CaBot to spell out its thought process and not focus solely on reaching a precise answer distinguishes it from any other AI diagnostic tool. It happens to be one of the few models that is designed to tackle a more intricate set of medical cases.
According to assistant professor of biomedical informatics in the Blavatnik Institute at HMS, Arjun Raj Manraj, they wanted to create an AI system that could alsogenerate a differential diagnosis and explain its detailed as well as nuanced reasoning at the level of an expert diagnostician. Manrai created Dr. CaBot along with a Harvard Kenneth C. Griffin School of Arts and Sciences doctoral student as well as a member of the Manrai lab, Thomas Buckley.
Although the system is not ready for use as yet in the clinic, Manrai, along with his team, has been offering demonstrations of Dr. CaBot at Boston-area hospitals. Now, Dr. CaBot happens to have a chance to prove itself by going head-to-head with an expert diagnostician in The New England Journal of Medicine’s famed Case Records of the Massachusetts General Hospital, which is also called clinicopathological conferences, or CPCs. It happens to mark the first time the journal is publishing a diagnosis that’s AI-generated.
The resulting medical case discussion, which was published on October 8 in NEJM, goes on to offer a window into the capabilities of Dr. CaBot, showcasing its usefulness when it comes to educators and students and also hinting at its potential as far as the physicians in the clinic are concerned. As the researchers continue to enhance Dr. CaBot, they hope that it is going to serve as a useful model for certain other medical-AI teams across the world.
100 years of medical cases
It is well to be noted that when we talk of the concept of CPC, it dates back to the late 1800s, when the physicians at Massachusetts General Hospital started making use of patient case studies for medical education tool. In 1900, Richard Cabot, the Mass General pathologist for whom Dr. CaBot is named, went on to formalize these as part of the curriculum for the HMS doctors-in-training. Since 1923, NEJM has been consistently publishing the cases as CPCs to teach the physicians how other physicians reason by way of complex cases.
Manrai said that the cases are pretty legendary, as they are known to be extremely challenging and filled with distractions as well as red herrings.
Apparently, each of the CPC happens to consist of a detailed presentation of the case from the doctors of the patients. Thereafter, an expert who is not involved in the case gets invited to give a presentation to colleagues at Mass General explaining the reasoning step-by-step and also providing a differential diagnosis before homing in on the most likely scenario. After that, the doctors of the patient reveal the actual diagnosis. The diagnostician’s write-up gets published in NEJM in addition to the case presentation.
The Oct. 8 NEJM article happens to include a typical case presentation with a carefully reasoned differential diagnosis coming from San Francisco Veterans Affairs Medical Center and the University of California, San Francisco’s expert diagnostician Gurpreet Dhaliwal, whom Manrai happens to describe as a real, modern Dr. House. Post that, Dr. CaBot’s differential diagnosis takes shape.
Apparently, Manrai and Buckley were encouraged to witness that although Dr. CaBot reasoned differently in the case when compared to Dhaliwal, it did reach quite a comparable final diagnosis.
From Dr. Cabot to Dr. CaBot
Manrai, during graduate school, went on to become fascinated by how CPCs go on to demystify the process that physicians make use of to arrive at a diagnosis. They did remind him of the mystery novels that he enjoyed growing up.
More recently, his lab, as well as others, has studied the precision of AI models for providing patient diagnoses. Manrai wondered if it were possible to design a system that could go further.
The core of Dr. CaBot happens to be OpenAI’s o3 large language reasoning model. In building the system, Buckley, who is a Dunleavy Fellow in the AI in Medicine PhD track of HMS, needed to augment o3 with novel capacities.
One is the ability of Dr. CaBot to efficiently search millions of clinical abstracts right from high-impact journals that helps it properly cite its work and to avoid factual hallucinations. Dr. CaBot can also search its brain pertaining to many thousand CPCs and make use of these examples so as to replicate the style of an expert diagnostician in NEJM. The team happens to be working quite closely with clinician collaborators at the Beth Israel Deaconess Medical Center, along with other Harvard-affiliated hospitals, so as to continue refining the system.
It is interesting to note that Dr. CaBot delivers two major products.
The first happens to be a roughly five-minute, narrated, slide-based video presentation of the case, wherein the system explains how it went on to reason through the possibilities so as to come to a diagnosis. The presentations happen to be surprisingly lifelike said Buckley.
During the team’s demonstrations, the realness of the narrated presentation looks to connect with the physicians, opined Manrai.
The other happens to be a detailed written version of the reasoning and diagnosis by Dr. CaBot.
Time to hit the Road
The researchers happen to be quite eager for physicians to engage with Dr. CaBot and at the same time also provide expert feedback. With regard to this, they are planning certain more demonstrations across local hospitals, and they went on to publish a paper that describes the system on a preprint server. They happen to witness the NEJM CPC as yet another opportunity for input.
As per the editor’s note on the CPC, Dr. CaBot’s AI-generated discussion has not been evaluated for correctness, and any factual errors that are present have been retained so that the reader can observe the positives and limitations of the system.  The note concluded that whether AI has a legitimate use in clinical decision-making is indeed completely up to the reader to gauge.