AI will generate convincing-sounding lists of affirming providers, detailed descriptions of their practices, or claims about their specialties—nearly all of which may be fabricated or outdated. The risk is especially high because you're often searching when you're vulnerable or time-sensitive, making it easy to skip the verification step that should always come before contacting a new provider.
Hallucination in AI systems means generating plausible-sounding but false or fabricated information. An AI might cite a provider name that doesn't exist, describe a clinic's policies it has never confirmed, or quote statistics that sound accurate but are entirely made up. The responses are coherent and confident, which makes hallucinations dangerous—they're easy to believe and hard to catch without fact-checking.
For LGBTQ+ healthcare research, hallucination is a significant risk because provider information changes constantly, networks vary by region, and the stakes of wrong information are high. You might spend time calling a clinic that doesn't exist, or assume a provider offers services they've never advertised, only to discover the error after traveling or waiting weeks for an appointment.
AI models like ChatGPT and Claude are trained to generate text that sounds plausible based on patterns in their training data. They're not connected to the internet (unless explicitly told to use search) and don't "know" what's current. When you ask "What gender-affirming clinics operate in Portland, Oregon?", the model generates text that follows the pattern of clinic descriptions from its training data—clinic name, address, services offered, insurance accepted. If its training data included a clinic that's now closed, it might still describe it in present tense. If the training data never mentioned a real clinic, the model might fabricate one rather than saying "I don't know."
Models are also overconfident. They don't say "I'm not certain" the way humans naturally would. They commit to false information with conviction. A hallucinated provider description sounds as authoritative as a real one because the model doesn't distinguish between "this was in my training data" and "I generated this pattern based on partial information."
Specific medical credentials are particularly prone to hallucination. The AI might say "Dr. X specializes in trans healthcare and has published on testosterone therapy" when that doctor never wrote about that topic. Phone numbers get fabricated—the format looks right but the number doesn't exist. Clinic policies get invented: "Most informed consent clinics require three therapy sessions before starting HRT" might be made up entirely, borrowed from unrelated sources, or represent patterns from the model's training data that don't reflect current practice.
Insurance information is frequently hallucinated. "Planned Parenthood clinics accept Medicaid in all states" is false (eligibility varies state-to-state), but a general model might state it confidently because its training data mentioned Planned Parenthood offering care and mentioned Medicaid coverage, without accurately connecting them.
The most dangerous hallucinations are specific-sounding claims that seem credible: "The Informed Consent Model for HRT is defined in the WPATH Standards of Care version 8 as requiring..." followed by a made-up definition. Real WPATH standards exist, so the reference feels legitimate, but the claimed definition might be wrong.
Always cross-check provider research with primary sources. If the AI recommends a clinic, search for it directly (Google Maps, clinic website, insurance directory). Call to confirm services, credentials, and current insurance acceptance. If the AI cites a specific document (WPATH standards, a study, a policy), find the actual document and verify the quote. AI often paraphrases sources and introduces errors in the paraphrase.
Use verification-focused tools like Perplexity AI when researching providers. Perplexity shows sources alongside its answers—you can see where information came from and fact-check immediately. When using ChatGPT or Claude, explicitly ask: "Please cite your sources for this information" or "Where did you get this information?" Some models will admit uncertainty, though others will still generate citations for hallucinated facts.
Cross-reference multiple sources. If you're researching HRT protocols, ask ChatGPT, then ask Claude, then check actual medical guidelines (ENDOCRINE SOCIETY, WPATH). If they largely agree, you've found solid ground. If they conflict, the conflict signals areas where hallucination is likely.
Be skeptical of extremely specific numbers without sources: "67% of clinics in the Midwest accept uninsured patients" sounds authoritative but might be fabricated. Vague citations are also risky: "Studies show that..." followed by no actual study link. Advice that contradicts what you've heard from actual providers should trigger verification—if your doctor said something different, trust your doctor, not the AI.
Most dangerous: don't use AI as your sole source for medical or clinical information. Use it as a research accelerator to generate questions and possible leads, then verify everything through official channels: clinic websites, provider directories (LGBTQ Health Center Finder, HRC's resource directory), and direct provider contact.
Try this: Ask ChatGPT to recommend three gender-affirming clinics in your area and describe their HRT approaches. Then independently search for those clinics online. For at least one recommendation, call and ask: "Do you offer the services and protocols this AI description mentioned?" Note where the AI was accurate and where it hallucinated. This exercise trains you to spot patterns in AI errors before they affect real decisions.
Peri can explain this concept, give practical examples, help you decide whether it applies to your situation, or recommend a journey if appropriate.
Explore related journeys or tell Peri what you're working through.