015: AI Differential Diagnosis Smackdown

Link to Youtube video

AI Differential Diagnosis Smackdown: ChatGPT vs. Claude vs. Meta AI

Hey everyone, it's Russell, your resident tech nerd and oral surgery geek. This week, I wanted to put some of the big AI language models to the test with a fun little experiment.

The contenders: OpenAI's ChatGPT. Anthropic's Claude3 Opus. Meta AI's LLaMA-3.

The challenge: Given the same photo of a lip lesion and identical prompts, how well can each AI generate a differential diagnosis list?

The lip lesion: Left lower lip, 0.75cm, irregular Pink, firm, mobile, non-tender, non-ulcerated Becoming bothersome due to size.

My initial DDx: Traumatic fibroma (most likely) Mucous cyst (common in this area) Lipoma (less likely based on appearance)

Results: ChatGPT: Impressive! Ranked fibroma/lipoma as most likely, generated a solid 9-item DDx list with descriptions. Even knew my specialty and suggested next steps.

Claude: Also very strong. 5-item DDx with great descriptions, emphasized biopsy for definitive diagnosis. Mentioned referral to OMS or ENT.

Meta AI: Interesting! Seemed to start generating a good list (benign and malignant possibilities) but then apologized and deleted its response. Possible guardrails around medical advice?

My take: While not a substitute for clinical judgment, I'm impressed by the AI's performance in this niche scenario. As capabilities expand and HIPAA/security concerns are addressed, I believe AI will become an increasingly valuable tool in our practices. Exciting (and a bit scary) to think about!

What do you think about AI's potential impact on dentistry and OMFS? I'd love to hear your perspective!

As always, feel free to reach out at [email protected]. Talk to you next week!

P.S. For those following along on YouTube, I've included a video of the AI interactions. Check it out and let me know what you think!