SPEC: Instruct Model
Definition
[Definition] An instruct model is a base model that has been further trained (via Supervised Fine-Tuning and/or RLHF) to understand and follow natural language instructions reliably. It responds to user requests rather than just completing text. Also called a "chat model" or "assistant model."
The Gap Between Base and Instruct
A base model trained to predict the next token knows a lot — but doesn't know it's supposed to be helpful. Fine-tuning on instruction-response pairs bridges this gap:
Base model sees: "Summarize this article: [article]..."
→ continues generating more article text
Instruct model sees: "Summarize this article: [article]..."
→ generates a concise summary
How Instruct Models Are Created
Step 1: Supervised Fine-Tuning (SFT)
- ▸Collect a dataset of (instruction, ideal response) pairs
- ▸Fine-tune the base model to reproduce the ideal responses
- ▸Dataset is human-written or GPT-generated and human-verified
- ▸Result: model learns the instruction-following format
Step 2: RLHF / Preference Optimization (Optional but Standard)
- ▸Human raters compare model outputs and rank them
- ▸Train a reward model to predict human preferences
- ▸Use PPO (or DPO, ORPO) to optimize the model toward higher-reward outputs
- ▸Result: model becomes more helpful, honest, and harmless
Instruct Format (Chat Template)
Most instruct models use a structured prompt format:
<|system|>
You are a helpful assistant.
<|user|>
What is photosynthesis?
<|assistant|>
Photosynthesis is the process by which plants...
Each model family has its own chat template (ChatML, Llama-3, Mistral, Gemma, etc.)
Key Capabilities Gained Through Instruction Tuning
| Capability | Description |
|---|---|
| Instruction following | Executes explicit user requests |
| Multi-turn conversation | Maintains context across turns |
| Formatting | Follows "respond in JSON", "use bullet points" |
| Role-playing | Adopts personas when instructed |
| Refusals | Declines harmful or out-of-scope requests |
| Conciseness | Answers the question without rambling |
Quality Dimensions of Instruct Models
| Dimension | Description |
|---|---|
| Helpfulness | Does it actually answer the question? |
| Harmlessness | Does it avoid dangerous outputs? |
| Honesty | Does it express uncertainty appropriately? |
| Instruction adherence | Does it follow all constraints in the prompt? |
| Format compliance | Does it match requested output format? |
Popular Instruct Models
| Model | Base | Notes |
|---|---|---|
| GPT-4o | GPT-4 (base) | OpenAI, closed |
| Claude 3.5 Sonnet | Claude (base) | Anthropic, via API |
| LLaMA 3.1 Instruct | LLaMA 3.1 | Meta, open weights |
| Mistral Instruct | Mistral | Open weights, efficient |
| Gemma 2 Instruct | Gemma 2 | Google, open weights |
Instruct Model vs. Chat Model
These terms are used interchangeably in practice. Some make a distinction:
- ▸Instruct model: single-turn task execution focus
- ▸Chat model: multi-turn conversational focus In practice, most modern models are both.
System Prompt Role
Instruct models accept a system prompt that shapes their behavior:
- ▸Persona definition: "You are an expert data scientist"
- ▸Behavioral constraints: "Always respond in French"
- ▸Safety rules: "Do not discuss competitors"
- ▸Tool/context injection: "You have access to the following tools: ..."
Tradeoffs of Instruction Tuning
[Key Insight] Instruction tuning makes models dramatically more useful for real-world tasks, but it comes with tradeoffs — the fine-tuning process can reduce raw capability compared to the base model, and safety-focused training may cause the model to refuse valid requests.
| Pro | Con |
|---|---|
| Much more useful for users | Can reduce raw capability vs. base |
| Safer behavior | May be overly cautious / refuse valid requests |
| Consistent format | May lose some creative/unexpected completions |
| Better at following constraints | Alignment tax on some benchmarks |
Related Concepts
- ▸Base Model, Fine-Tuning, RLHF, System Prompt, User Prompt, Alignment, SFT