Periagoge
Concept
2 min readself knowledge

Tokens and Word Counts: Why AI Has Limits and How to Work Within Them

A token is roughly a word, but AI counts them in a way that doesn't map perfectly to human word counts—typically 1,000 words ≈ 1,300 tokens. Each AI conversation has a hard limit (your model's context window), and both what you send and what you get back eat into that budget, so knowing your token efficiency prevents conversations from crashing mid-thought.

Hypatia
Why It Matters

Think of tokens like coins at an arcade. Each "coin" buys you a small piece of text—roughly 1 token equals 4 characters or about 0.75 words. When you hit your token limit, the AI stops mid-sentence, like you ran out of coins and the machine shuts down.

Why This Matters

Every AI tool has a maximum length it can handle in one conversation. Claude can handle about 100,000 tokens (roughly 75,000 words). ChatGPT maxes out lower depending on which version you're using. This affects your creative work because if you're writing a long short story or asking for detailed feedback, the AI might stop before finishing.

It's especially tricky because the limit counts both what you send to the AI AND what it sends back. So if you paste in a 2,000-word chapter plus your question, that eats into what the AI can give you in response.

How This Affects Creative Projects

Let's say you're working on a novel outline and you feed the AI your existing 5,000-word worldbuilding document, then ask for help expanding a key scene. The AI now has less "room" to write back. It might give you 1,500 words instead of the 3,000 you wanted.

The practical solution: work in smaller chunks. Instead of pasting your entire manuscript, share one chapter at a time. Instead of asking "revise this whole novel," ask to revise one scene. This keeps you under limits and usually produces better results anyway (remember prompt chaining?).

The Light You'll See at the End

Tools keep improving their limits. But the real skill is learning to work within constraints—it actually forces better, more focused creative decisions. Your 2,000-word scene is better than a sprawling 10,000-word mess anyway.

Try this: Next time you paste something into an AI, do a quick word count. Then ask for a response. Check if it seems cut off by looking for incomplete sentences or abrupt endings. If it is, break your request into smaller pieces and ask again.

Helpful guides
Hypatia
Daily Life & Decisions
Related Concepts
Peri
Questions about Tokens and Word Counts: Why AI Has Limits and How to Work Within Them?

Peri can explain this concept, give practical examples, help you decide whether it applies to your situation, or recommend a journey if appropriate.

Ready to work on Tokens and Word Counts: Why AI Has Limits and How to Work Within Them?

Explore related journeys or tell Peri what you're working through.