What is a context window? Tokens, max tokens, context length, full windows, memory, MCP

A context window is the maximum amount of text, measured in tokens, that a language model can consider at once, including your prompt, conversation history, documents and the model's own response. When it fills up, older content must be dropped or summarised. I build RAG systems that exist largely because of context limits, so this page covers context windows, tokens, max tokens, input and output, context length, full windows in Claude Code, agent memory, masking, MCP and very large windows.

What is a context window?

A context window is the model's working memory: the total number of tokens it can process in a single request, covering system instructions, conversation history, retrieved documents, tool results and the generated answer.

Also asked as: what is a context window? unlocking llm secrets

Models cannot see anything outside the window, so long conversations or documents must be trimmed, summarised or retrieved selectively. Window sizes have grown from a few thousand tokens in early models to hundreds of thousands or millions today. Larger windows allow whole documents or codebases, but cost and latency grow with input size, and models may attend less reliably to details buried in the middle. Good systems choose what to include rather than filling the window by default.

Token vs context window: what is the difference?

A token is a unit of text, often a word or part of a word, that models read and write. The context window is the maximum number of tokens a model can handle at once. Tokens are the units; the window is the capacity.

Also asked as: how grok 4's 2 million token context window changes everything we know about ai

In English, one token averages about three to four characters, or roughly three quarters of a word, though other languages, including many Indic scripts, can use more tokens per word. Tokenizers differ between models, so the same text can count differently. Use the provider's tokenizer or token-counting API to measure prompts accurately. Pricing and limits are counted in tokens, so understanding tokenisation helps estimate cost and avoid hitting window limits unexpectedly.

What is the difference between max tokens and the context window?

The context window is the model's total capacity for input plus output. Max tokens, or max output tokens, is a request parameter that caps how many tokens the model may generate in its response.

Also asked as: difference between max tokens and context window

Your input tokens plus the max output setting must fit within the context window, and many models also have a separate fixed limit on output length, smaller than the window. Setting max tokens too low truncates answers mid-sentence; setting it high does not force long answers but reserves room. For reasoning models, internal reasoning tokens may also count toward output limits, so allow generous values. Check each model's documented context and output limits.

Does the context window include input and output tokens?

Yes. The context window covers both: everything you send, including the system prompt, history, documents and tool results, plus the tokens the model generates. Together they must fit within the window.

Also asked as: does context window include input and output tokens

In multi-turn conversations, previous outputs become part of later inputs, so windows fill faster than people expect. Reasoning models may generate hidden thinking tokens that also occupy space during the request. Applications manage this by trimming old turns, summarising history, or retrieving only relevant past messages. Watch the usage fields returned by APIs, which report input and output tokens per request, to understand consumption and cost precisely and to catch prompts that are growing too large.

What is the difference between context window and context length?

Usually they mean the same thing: the maximum number of tokens a model can process at once. Some people use context length for the actual length of a given prompt and context window for the maximum capacity.

In model specifications, context length often refers to the maximum sequence length the model was trained or configured to handle, such as 128,000 tokens. In local inference tools like llama.cpp or Ollama, you set a context length parameter that may be smaller than the model's maximum, to save memory, since the KV cache grows with it. Window size can also refer to sliding window attention, where each token attends only to nearby tokens, a different concept.

What happens when the context window is full in Claude Code?

Claude Code automatically compacts the conversation when the context approaches its limit, summarising earlier messages so work can continue. You can also run /compact yourself with optional instructions about what to keep, or /clear to start fresh.

Also asked as: what happens when context window is full claude code

To avoid losing important details, keep durable instructions in a CLAUDE.md file, which Claude Code loads into every session, and break large tasks into focused sessions. Ask Claude to write plans or progress notes to files, so state survives compaction. Use subagents for broad searches so their large outputs stay out of the main context. Monitor context usage indicators, and compact at natural breakpoints, such as after finishing a feature, rather than mid-task.

How do you architect AI agent memory that survives context window limits?

Separate memory into layers: the current context for the active task, a running summary of the session, and long-term memory stored outside the model, such as files, databases or vector stores, retrieved when relevant.

Also asked as: how to architect ai agent memory that survives context window limits

Practical patterns: write structured notes and decisions to files the agent rereads; summarise completed steps and drop raw details; store facts and preferences with embeddings and retrieve them by similarity; keep task state, such as a to-do list, in a durable store; and use subagents with their own contexts for large explorations, returning only conclusions. Test memory by restarting sessions and checking whether the agent recovers its goals correctly.

Why do LLMs need masking and a context window during inference?

Causal masking ensures each token attends only to earlier tokens, matching how the model was trained to predict the next token. The context window bounds how many tokens attention covers, because memory and compute grow with sequence length.

Also asked as: why do we need masking and context window during inference in llms

During inference, the model generates one token at a time, and the KV cache stores keys and values for previous tokens so they are not recomputed; masking keeps attention causal within this cache. Padding masks also tell the model to ignore padding when batching sequences of different lengths. The window limit exists because positional encodings were trained for certain lengths and because attention cost and KV cache memory grow with every token.

MCP vs context window: how do they relate?

They are different things. The context window is the model's token capacity per request. MCP, the Model Context Protocol, is a standard for connecting AI applications to tools and data. MCP results consume context window space when returned to the model.

Each connected MCP server adds tool descriptions to the prompt, and every tool result, such as a file's contents or a database query, adds tokens. Many servers with verbose tools can fill the window quickly, leaving less room for reasoning. Good practice: enable only the servers you need, design tools that return concise, filtered results, paginate large outputs, and summarise before passing data on. MCP extends what a model can access, not how much it can hold.

Do huge context windows, such as 2 million tokens, change everything?

Very large windows, such as the 2-million-token contexts offered by some models, make it possible to load whole codebases, books or long histories at once, which simplifies some applications. They do not remove the need for retrieval and context design.

Long inputs cost more, increase latency and can reduce accuracy, as models may miss details buried in vast contexts. Benchmarks that hide facts in long documents show that performance varies by model and position. Retrieval still wins for large, changing knowledge bases, access control and citations. Large windows work best for tasks needing broad context at once, such as analysing a full contract or repository, combined with careful prompting and verification of key details.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail