3. LLMs, Tokens and Context
3.1 What is an LLM?
LLM means Large Language Model.
LLMs process tokenized input and generate tokenized output.
They can be used for:
- Text generation
- Code generation
- Summarization
- Translation
- Classification
- Reasoning
- Tool use
3.2 Tokens
A token is a unit processed by the model.
Tokens are not always identical to words.
For example, a tokenizer may split text into:
"OpenCode"as one or multiple tokens depending on the tokenizer.
Code is tokenized too.
def add(a, b):
return a + b3.3 Context
Context is the information available to the model for a particular interaction.
A coding agent may work with:
System instructions
+
User request
+
AGENTS.md
+
Relevant source files
+
Tool results
+
Skill instructions
+
Previous conversation3.4 Context window
The context window limits how much information can be processed in one model interaction.
More context is not automatically better.
Bad:
Entire repository
+
All logs
+
All documentation
+
Unrelated filesBetter:
Task
+
Project rules
+
Relevant files
+
Relevant tests
+
Relevant skill3.5 Tokens, cost and latency
Depending on the provider/model, more tokens can affect:
- Cost
- Latency
- Context pressure
Exact pricing and limits vary by provider and model.
Always check the provider's current pricing and model documentation.
3.6 Context compression
A useful strategy is to preserve:
- Decisions
- Requirements
- Relevant errors
- Important constraints
- Relevant code
and remove:
- Repeated explanations
- Unrelated logs
- Duplicate documentation
- Irrelevant history
Exercise
Take this request:
Fix the login bug.
Rewrite it so an AI coding agent receives:
- Goal
- Scope
- Constraints
- Verification requirements
- Expected output