AI Prompt EngineeringModuleLesson 01
Tokens & Context Windows
15m
Beginner
+50 XP
0% Done
When you talk to ChatGPT or Claude, it feels like they are reading your words, right? Like they understand English just like you do.
This is a common illusion.
AI models cannot read letters or words. They don't know what an "A" or a "B" is. They only understand numbers.
To bridge this gap, AI uses a system called Tokenization. A token is a chunk of text. Think of it as a piece of a puzzle.
When you send a prompt, the AI's first job is to chop your sentence into these tiny puzzle pieces (tokens), convert each piece into a number, and then process the numbers.
See how AI breaks words into chunks.
Open ChatGPT or Claude and try this exact prompt: 'Write a 10-word sentence where every single word starts with the letter S.' Count the words it gives you. Did it exactly follow the instruction?
Why do LLMs sometimes struggle with tasks like spelling or counting letters?
Tokens are the fundamental building blocks of AI communication. By understanding that AI reads chunks of text as numbers, you can avoid common pitfalls related to word counts, spelling, and strict formatting.