In-Context Learning
An astonishing emergent property of large language models that allows them to instantly grasp new rules, unknown words, or unique formats directly from the ongoing conversation text without altering neural network weights or programming.
1. Concept Overview & Systemic Problem
Consider how you learn to play a new board game with friends: there’s no brain surgery or alteration of neurons. You’re simply told two straightforward rules: “The red piece beats the blue, and the green gives two moves.” Your brain instantly adapts to these rules right during the game.
In-Context Learning is precisely this capability of modern large language models. They have learned to think on the fly: you convey an unusual rule, an invented word, or a unique corporate cipher directly in the message — and the model immediately restructures its logic to meet your temporary requirements.
Practical analogy: the simplest way to tailor a model for your unconventional tasks without reaching out to programmers.
2. Architectural Taxonomy & Mental Model
┌─────────────────────────────────────────────────────────────┐
│ TWO WAYS TO TRAIN ARTIFICIAL INTELLIGENCE │
├─────────────────────────────────────────────────────────────┤
│ 🏋️ Traditional Training (Fine-Tuning): │
│ • Changing numerical coefficients (weights) of the model │
│ • Requires powerful GPUs, thousands of datasets, and hours│
│ • Permanently embedded in the model's memory │
├─────────────────────────────────────────────────────────────┤
│ ⚡ In-Context Learning (Instant Adaptation in Chat): │
│ • Model weights are NOT changed (they are fixed) │
│ • Learning occurs through the Self-Attention mechanism │
│ • Works instantly: you provide an example ➔ receive a result│
│ • Disappears after closing the chat tab │
└─────────────────────────────────────────────────────────────┘
3. How Beginners Can Leverage This Superpower
01. Creating Your Own Encryption or Status System
You can teach the model your company’s abbreviations in a single message:
“In our support service, we have the following status codes: [GREEN] — client satisfied, closing the request. [YELLOW] — awaiting bank response. [RED] — urgent complaint to management. Now assign these statuses for the following messages: [List of emails]”.
02. Training on a Completely New Report Format
If your boss requires a specific word order in a report that is not used anywhere else in the world:
“Here’s how I format the daily summary: [Showed one example]. Create a similar report for today’s events.”.
03. Adapting to Internal Corporate Slang
If your team has specific department names or internal jokes, just list them at the beginning of the conversation — the AI will communicate like your longtime colleague.
FAQ: In-Context Learning
Related terms
Zero-Shot vs. Few-Shot (The Power of Examples)
A comparison of two fundamental prompting techniques. Zero-Shot involves a direct query without examples, while Few-Shot provides the model with several samples of the desired response before executing the task for precise calibration of format and tone.
Tokens Explained (How Many Words in a Token)
The fundamental unit of measurement for text in language models. This entry explains how words are broken down into tokens, why this impacts query costs, and why Ukrainian words consume more tokens than English ones.
Context Window Size (Current Conversation Memory)
The maximum amount of text (in tokens) that a language model can simultaneously retain in memory during an ongoing conversation. It determines the length of documents that can be loaded at once without loss of content.