How LLMs Work (Tokens, Context Windows & Temperature)

Understanding the Engine Behind Modern AI

Large Language Models (LLMs) like GPT-4, Claude, and Gemini do not "think" or feel like humans. They are next-token prediction engines trained on massive amounts of text.

What is a Token?

A token is the fundamental chunk of text that an LLM reads and generates.

• 1 token $\approx$ 4 characters of English text or 0.75 words.
• The phrase "Prompt Engineering" is broken down into 2-3 tokens (e.g., ["Prompt", " Engineer", "ing"]).

Context Window: The Short-Term Memory

The context window is the maximum number of tokens an LLM can process in a single conversation (including your input prompt and its output response).

• Standard models have context windows ranging from 8,000 to 2,000,000+ tokens.
• If your conversation exceeds the context window, early messages get forgotten!

Temperature: Controlling Creativity

Temperature is a parameter between 0.0 and 1.0 (or 2.0) that controls randomness:

Low Temperature (0.0 - 0.2)  ---> Precise, deterministic, factual (ideal for code & math)
Medium Temperature (0.5 - 0.7) ---> Balanced, natural conversation
High Temperature (0.9 - 1.2) ---> Creative, imaginative, diverse (ideal for brainstorming)