The rooms at Siriraj x MIT Hacking Medicine 2026 were full of doctors, pharmacists and health researchers who had already started using ChatGPT. The curiosity was high. The experimentation was real. What was missing wasn't motivation. It was structure.
Our team mentored hackathon teams over the weekend and spent the conference days talking through AI use cases with clinicians. The gaps we saw weren't about medical knowledge or technical aptitude. They were practical: how these tools actually work, where data can safely go, and how to know when an answer is trustworthy.
That matches the numbers. In the American Medical Association's 2026 survey of nearly 1,700 physicians, 81% already use AI professionally, yet 88% say safety and efficacy validation is critical and 86% say the same of data privacy [1].

Here's what we think every clinician working with AI should understand, and what most workshops skip.
1. A mental model of how LLMs fail
Large language models don't retrieve facts. They predict the next most probable words based on patterns in their training data. That means they can produce confident, fluent, completely wrong answers.
The problem isn't that clinicians don't know this in theory. It's that the experience of using ChatGPT doesn't reinforce it. The output sounds authoritative. The formatting looks professional. "It sounded right" becomes the default validation, and that's dangerous in a clinical context.
Understanding that these models are probabilistic, not deterministic, changes how you use them. You design workflows that assume the model might be wrong, and you build in checks before any output touches a decision.
2. Knowing where data can and can't go
Many clinicians we spoke with were using consumer chatbots for work tasks. Some knew about the privacy risks. Fewer knew there was an alternative.
The rule is simple: patient data never goes into a general-purpose tool. Not into a free chatbot, and not into any service that might use your input for training or store it outside your control. In Thailand, health information is sensitive personal data under the Personal Data Protection Act, which sets a higher bar for collecting and using it [2].
Enterprise setups (Azure OpenAI, Google Vertex AI, AWS Bedrock and others) offer contractual terms on how your data is handled. Consumer tools don't. Training opt-outs help, but they aren't enough on their own. If it's patient data, it belongs in a system your institution controls, with an audit trail.

3. Retrieval vs memory
A request we hear often: "I want the AI to know our hospital's guidelines."
The instinct is to assume the model can memorise your protocols. It can't, not reliably, and not without mixing them up with everything else it was trained on.
The better approach is retrieval-augmented generation (RAG): the system searches your own documents first, then uses those passages to ground its answer. You're not asking the model what it "knows" about liver disease. You're asking it to summarise the relevant section of your internal guideline, and to show you where it found it. In a 2024 benchmark of medical question answering, adding retrieval improved accuracy across six models by up to 18% [3].

This also solves the cross-contamination problem. A well-designed system keeps knowledge partitioned by domain, so a query about liver function doesn't pull content from a lung protocol.
4. How to test a tool before trusting it
No one should deploy an AI assistant without testing it first. But most clinicians don't have a testing habit, because they don't have a method.
Here's a simple one: build a small question bank of ten to twenty questions that represent real use cases. Run them through the tool. Grade each answer as pass, warn or fail.
Then test it through personas. What does it say to a doctor? A nurse? A pharmacist? A patient? Does it refuse questions it shouldn't answer? Does it escalate appropriately?

This requires a structured hour and a willingness to treat the tool like any other clinical instrument: something you validate before you rely on it.
5. Governance is the real blocker, not the model
The technology works. What stops most healthcare organisations from using it isn't the model. It's governance.
Compliance teams want guarantees that LLMs can't give: zero risk, deterministic behaviour, and liability someone else will carry. None of that exists. The World Health Organization's guidance on large multi-modal models makes the same point from the other side, calling for these tools to be built for clearly defined tasks, with clinicians involved from the start [4]. As WHO Chief Scientist Dr Jeremy Farrar put it: "We need transparent information and policies to manage the design, development, and use of LMMs." [4]
The answer isn't to wait for a perfect model. It's to decide what the system refuses to do, who reviews its outputs, and how you keep an audit trail.

6. The underrated win: a place to ask "stupid questions"
One theme we keep hearing from clinicians is the psychological value of a private AI assistant.
Clinicians are expected to know everything. Asking a colleague a basic question can feel like admitting ignorance. A private assistant gives them somewhere to ask without judgment.
That's not a trivial benefit. It closes knowledge gaps in real time, reinforces learning, and reduces the load of staying current. The tool doesn't replace expertise; it supports it. Or, in the AMA's words, AI should be "designed to enhance, not replace, physicians" [1].
What a good AI workshop for clinicians should cover
Most AI workshops for healthcare are either too technical (how transformers work) or too abstract (the future of medicine). Neither helps someone use AI safely tomorrow.
A practical workshop should cover:
- A mental model: how LLMs generate output, why they hallucinate, what that means for validation.
- Data rules: consumer vs enterprise, what counts as patient data, where the legal boundaries are.
- Hands-on practice: using the tool with their own non-clinical documents (guidelines, protocols, research summaries).
- Testing habit: building a question bank, grading outputs, testing across personas.
- Governance basics: scope, human review, audit trails.
The goal isn't to turn clinicians into AI engineers. It's to give them enough structure to experiment safely, and enough judgment to know when to stop.
We build this
Pulsar runs practical AI workshops and builds governed assistants for teams in healthcare and other regulated fields. If your organisation is experimenting with AI but hasn't figured out the governance piece, we should talk.
No hype. No one-size-fits-all demos. Just tools that work, and training that sticks.
