Most AI systems can only 'see' a limited window of past conversation at once, meaning a medication change from three months ago might disappear from the system's working memory even though you remember it clearly. This technical limitation matters in practice—you often need to re-introduce key context.
A context window is the amount of text an AI model can "see" at once—the complete conversation history plus any documents you provide. For Claude 3.5, that's 200,000 tokens (roughly 150,000 words). For GPT-4, it's 128,000 tokens. This seems unlimited, but in caregiving workflows, it becomes a real constraint that affects care continuity and decision quality.
Here's the practical consequence: you're managing your elderly parent's care across 18 months. You upload all medical records, appointment summaries, medication histories, caregiver notes, and insurance documents to an AI conversation. The total is 80,000 tokens. You start asking questions, and the conversation grows. By token 100,000, you've still got context. By token 150,000, you're pushing limits. Most LLMs begin losing earlier information (a phenomenon called context degradation) before they hit absolute limits. Information you provided at the start of the conversation becomes less accessible or is weighted less heavily by the model when making decisions or synthesizing new information.
Imagine you uploaded appointment notes from six months ago mentioning your parent's allergy to penicillin. You've had 30 conversations since then, the context window is full, and a new doctor prescribes amoxicillin (a penicillin-based antibiotic). If the allergy information is in the context but weighted too low due to decay, the AI might not flag the conflict. This is especially dangerous because the model doesn't tell you "I may have missed information." It confidently generates a response that appears complete.
The second issue: recency bias. Many LLMs weight recent tokens more heavily than earlier ones. In a long caregiving conversation, recent appointment notes are weighted more than foundational health history. This distorts synthesis—current symptoms become over-emphasized relative to chronic patterns that contextualize them.
First, segment conversations strategically. Don't maintain one infinite conversation for all caregiving AI work. Use separate conversations for different domains: one for medication coordination, one for appointment scheduling, one for dietary planning. This keeps context windows focused and prevents information decay from unrelated topics.
Second, version your information stores. Maintain a living master document (in Notion, Google Docs, or a proper medical records system) that's your source of truth. Upload it fresh to each new conversation rather than relying on persistent context. This eliminates decay and ensures each conversation has complete information at full weight.
Third, use summaries as context compression. Instead of uploading a 200-page medical record, use an AI to create a structured summary: "6-page executive summary of 18 months of care with medication timeline, allergies, recent lab results, and open action items." This compressed information fits comfortably in any context window, is less prone to decay, and is easier for the model to synthesize.
For advanced workflows, Retrieval-Augmented Generation (RAG) is the architectural solution. Rather than loading entire histories into context, RAG systems query relevant documents on-demand. You ask a question, the RAG system retrieves only the documents relevant to that question, and the model works with focused context. This eliminates context window pressure entirely.
Different models handle context differently. Claude 3.5 Sonnet excels with long contexts and shows less decay. GPT-4 is more prone to recency bias in very long conversations. Gemini 1.5 Pro has a 2-million-token window but exhibits different information weighting patterns. Understanding these nuances means choosing the right model for context-heavy caregiving tasks.
Try this: Create a 20-conversation roleplay scenario in Claude. Upload a medical history document (5,000 tokens), then have 15 conversations asking questions. In conversation 16, ask a question that requires information from the initial upload. Notice whether the model still has accurate recall of foundational details or if information has become fuzzy. Then, start a fresh conversation with the same document and ask the same question. Compare response quality. This demonstrates why conversation segmentation matters for caregiving accuracy.
Peri can explain this concept, give practical examples, help you decide whether it applies to your situation, or recommend a journey if appropriate.
Explore related journeys or tell Peri what you're working through.