#36 General Purpose LLMs vs Specialised AI- Is Bigger always better? (OpenEvidence vs OpenAI)

Do big frontier models outperform narrow AI tools? Here we look into the healthcare domain, where a paper titled "General-purpose large language models outperform specialized clinical AI tools on medical benchmarks" was making headlines. The paper seemed to suggest that frontier LLMs from OpenAI and Gemini appeared to outperform more specialised clinical AI tools. However, there is more than meets the eye here. In this episode Dev and Doc deep dive into the fascinating debate of whether generalised LLMs do indeed outperform smaller specialised models.👋 Hey! If you are enjoying our conversations, reach out, share your thoughts and journey with us. Don't forget to subscribe whilst you're here :)2:25 intro - Does the convention still hold?7:05 what about coding? How general is a task8:38 data availability - Healthcare's data dichotomy11:14 OpenEvidence Nature paper start12:33 Context from a Doctor before AI21:58 paper break down - benchmarks methodology31:59 how to design clinical rubrics34:22 OpenEvidence rebuttal37:20 Conclusion & discussion46:59 how to improve the researchTo support us buymeacoffee.com/devanddoc👨🏻‍⚕️Doc - Dr. Joshua Au Yeung - https://www.linkedin.com/in/dr-joshua-auyeung/🤖Dev - Zeljko Kraljevic https://twitter.com/zeljkokrYT - https://youtube.com/@DevAndDocSpotify - https://podcasters.spotify.com/pod/show/devanddocApple - https://podcasts.apple.com/gb/podcast/dev-and-doc-ai-for-healthcare-podcast/id1751495120Substack - https://aiforhealthcare.substack.com/For enquiries - 📧[email protected]🎞️Editor- Dragan Kraljević https://www.instagram.com/dragan_kraljevic/🎨Brand design and art direction - Ana Grigorovici https://www.behance.net/anagrigorovici027d