AgentAlley

Skills · Data & AI

Fine-Tuning Method Selection

Unverified●32/40

Decide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing between preference-optimization and reinforcement methods.

Originally by wshobson · MIT

Claude Code◐PartialHas SKILL.md but declares no allowed-tools — Claude Code will ask for permission each time
Cursor◐PartialPlain prose you can paste in — but no Cursor rules file
Codex◐PartialPlain prose you can paste in — but no AGENTS.md
Gemini CLI◐PartialPlain prose you can paste in
Copilot◐PartialPlain prose you can paste in — but no Copilot instructions file
npx agentalley add finetuning-method-selection

This command does not work yet — the CLI is still being built. Until then, use Raw in the reader below to take the file.

Who is stuck, and on what

Decide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing between preference-optimization and reinforcement methods.

The whole source

No sign-in, no blur, nothing truncated
finetuning-method-selection/SKILL.md215 lines7.7 KBRawView on GitHub
Frontmatter — 2 properties
namefinetuning-method-selection
descriptionDecide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing between preference-optimization and reinforcement methods.
1---
2name: finetuning-method-selection
3description: Decide whether to fine-tune at all, and route to the right method (SFT, DPO/ORPO/KTO, GRPO/RLVR, continued pretraining) and base model. Use when starting any fine-tuning effort, when unsure whether RAG or prompting would suffice, or when choosing between preference-optimization and reinforcement methods.
4---A5 — No allowed-tools declared — no way to tell what this skill may touch
5 
6# Fine-Tuning Method Selection
7 
8This is the router skill for the fine-tuning
9lifecycle: it decides whether fine-tuning is the
10right tool at all, and if so, which method and
11which base-model size class. Every other skill
12in this plugin assumes this routing already
13happened — start here before opening
14`lora-qlora-recipes`, `preference-optimization`,
15or `grpo-rlvr-training`.
16 
17## When to Use This Skill
18 
19- Starting any fine-tuning effort, before a
20 framework or base model has been chosen.
21- Unsure whether RAG or prompt engineering would
22 solve the problem more cheaply than training.
23- Choosing between preference optimization (DPO
24 family) and a reinforcement method (GRPO/RLVR)
25 for the same underlying task.
26- Sizing a candidate model/method combination
27 before committing to a run.
28 
29## Quick Reference
30 
31| Situation | Route |
32|---|---|
33| Facts change often (prices, docs, news) | RAG, not fine-tuning |
34| Desired behavior still being figured out | Prompt engineering |
35| Stable domain knowledge, ≥500MB text | CPT then SFT — see Off-Ramps First |
36| Have input/output demonstrations | SFT — see `lora-qlora-recipes` |
37| Have preference pairs or thumbs-up/down | DPO/ORPO/KTO — see `preference-optimization` |
38| Have a verifiable pass/fail signal | GRPO+RLVR — see `grpo-rlvr-training` |
39| No eval harness yet | Stop — see `eval-harness-first` |
40 
41## Off-Ramps First
42 
43Most requests that sound like "fine-tune this"
44are served better and cheaper elsewhere. Check
45these off-ramps before opening a training run:
46 
47- **Knowledge-bound and volatile** (the gap is
48 facts that change — prices, docs, current
49 events): route to RAG, not fine-tuning. A
50 fine-tuned model bakes in a snapshot; volatile
51 facts go stale immediately.
52- **Behavior-bound and shifting** (the desired
53 behavior is still being figured out, or
54 changes per request): route to prompt
55 engineering. Fine-tuning locks in a behavior;
56 don't lock in one that hasn't stabilized yet.
57- **Stable, dense domain knowledge**: this is
58 where continued pretraining (CPT) enters, sized
59 by how much domain text exists:
60 
61| Domain text volume | Route |
62|---|---|
63| <10MB | RAG only |
64| 10MB–500MB | RAG + fine-tune |
65| 500MB–10GB | CPT, then SFT |
66| >10GB | CPT required |
67 
68CPT learning rate ≈ **10% of the pretraining
69LR**. CPT is guidance-only in this plugin —
70sizing and LR guidance live here, but this
71plugin does not execute a CPT run.
72 
73## Method Router
74 
75Once the off-ramps are ruled out, this is the
76full decision tree (verbatim from the research
77this plugin is built on):
78 
79```
80New FACTS? volatile → RAG | stable+dense → CPT (LR ~10% of pretrain) → SFT
81New BEHAVIOR? shifting → prompt-engineering | stable:
82 demos → SFT (LoRA/QLoRA, all-linear, α=2r)
83 preference pairs → DPO (SimPO if length-bias, ORPO if memory-bound)
84 unpaired 👍/👎 → KTO
85 verifiable success → RLVR + GRPO (DAPO/GSPO/Dr.GRPO per failure mode)
86Deploy: FP8 (Hopper+) | NVFP4 (Blackwell scale) | AWQ (older) | GGUF+imatrix (edge)
87BEFORE ANY OF THIS: the eval harness must exist first.
88```
89 
90Read the tree top-down: answer "new facts or new
91behavior," then follow the branch that matches
92the data shape in hand (demos, preference pairs,
93thumbs up/down, or verifiable success/failure).
94The data shape picks the method — not the other
95way around.
96 
97### Worked Routing Examples
98 
99- *"Users want the assistant to follow our
100 support macros exactly."* Behavior is stable
101 and demonstrable from transcripts → demos →
102 **SFT**.
103- *"We have pairs of good/bad responses from
104 reviewer thumbs-up/down, unpaired."* → unpaired
105 signal → **KTO**, not DPO (DPO needs paired
106 preferences).
107- *"The model can already solve some of these
108 math problems and we can grade correctness
109 automatically."* → verifiable success signal →
110 **GRPO+RLVR**, and only after confirming the
111 model succeeds at least sometimes (see Key
112 Routing Facts below).
113- *"We want the model to know this week's
114 pricing page."* → volatile facts → **RAG**, no
115 training run at all.
116 
117## Key Routing Facts
118 
119- **Loss-function choice is low-leverage.** A
120 240-H100-run study found method choice worth
121 ~1 percentage point versus ~50 points for model
122 scale, and zero of 20 DPO variants beat vanilla
123 DPO. Don't spend a routing decision agonizing
124 over DPO-variant selection — spend it on
125 getting the data shape and scale right.
126- **DPO is for taste, GRPO+RLVR is for
127 reasoning.** Preference pairs that encode a
128 subjective judgment (tone, style, "which answer
129 is better") route to DPO. Tasks with a
130 verifiable pass/fail signal (math, code, tool
131 calls) route to GRPO+RLVR instead.
132- **RL is not the fix for a model that never
133 succeeds.** GRPO and other RL methods sharpen
134 an existing capability — they don't teach one
135 from zero. If the model doesn't yet understand
136 the task or output format, run SFT first; only
137 bring in RL once the model succeeds at least
138 sometimes.
139 
140### Common Routing Mistakes
141 
142- Reaching for fine-tuning to fix facts that
143 change weekly — that's a RAG problem, and
144 fine-tuning will just go stale faster than the
145 source data does.
146- Picking a DPO variant before checking whether
147 the actual bottleneck is data quality or model
148 scale — variant choice is the ~1pp lever, not
149 the ~50pp one.
150- Starting an RL run on a model that fails every
151 rollout — route to SFT first so RL has
152 something to sharpen.
153- Treating CPT as the default for "the model
154 doesn't know our domain" — check the data
155 volume thresholds first; under 500MB, RAG or
156 RAG+fine-tune iterates faster than a CPT run.
157 
158## Model Selection
159 
160Base-model choice is size-class first, family
161second, and it goes stale fast — so it lives in
162exactly one place: `references/model-catalog.md`.
163That file is the only place in this plugin (and
164in the DGX Spark ops plugin) that names a base
165model family. Neither this skill nor
166`references/memory-math.md` names one; both
167describe models by size class only (for example,
168"8B-class LoRA," not a model name).
169 
170The catalog is dated on purpose — model rankings
171turn over quarterly. It carries a "last verified"
172date and a refresh checklist. Before trusting a
173row, check that date; if stale, work the refresh
174checklist in the catalog before recommending a
175model from it.
176 
177**Precedence when the catalog and a method skill
178disagree:** the catalog's per-row Notes column
179states hardware/size-class *feasibility*, not a
180method recommendation — `lora-qlora-recipes`'s
181LoRA vs QLoRA vs Full FT table (routed by task
182shape) governs the actual method choice.
183 
184## Memory Feasibility
185 
186Before committing to a method, size it: total
187memory ≈ **params × dtype bytes + optimizer
188state + gradients + activations**. Work each
189term for the chosen dtype and method (full
190fine-tune, LoRA, or QLoRA) — worked worksheets
191and size-class examples live in
192`references/memory-math.md`.
193 
194On DGX Spark specifically, unified-memory
195behavior breaks the naive estimate (transient
196load peaks, `nvidia-smi` underreporting, thermal
197throttling on long runs). Once the
198`dgx-spark-ops` plugin is installed, defer
199Spark-specific feasibility calls to its
200`spark-memory-thermal-ops` skill rather than
201re-deriving them here.
202 
203## Related Skills
204 
205Once this skill has picked a method, hand off to
206the skill that executes it:
207 
208- `lora-qlora-recipes` — SFT via LoRA/QLoRA
209- `preference-optimization` — DPO, ORPO, KTO
210- `grpo-rlvr-training` — GRPO with verifiable
211 rewards
212 
213No method is selected before the eval harness
214exists — see `eval-harness-first`.
215 

Reviews

Installed this one?Write the first review and take the Trailblazer badge.

Reviews only open after a real install, so this is empty — and we leave it empty rather than invent one.

Alternatives

Also in Data & AI
Task Coordination StrategiesDecompose complex tasks, design dependency graphs, and coordinate multi-agent work with proper task descriptions and workload balancing. Use this skill when breaking down work for agent teams, managing task dependencies, or monitoring team progress.Data & AI · MIT◐◐◐◐◐●35/40Ebay Seller ToolsData & AI · MIT·····●34/40Tough Decision Advisor: Every Angle ConsideredHand in a decision you're stuck on. Get back a clear breakdown of every angle — the trade-offs, the risks, the blind spot, and a recommended path.Data & AI · Business · MIT license●····●32/40DHDNA Profiler — Cognitive Pattern ExtractionPaste any email, proposal, or note someone wrote, and get back a plain-language read on how they think, what drives their decisions, and how they communicate.Data & AI · Research & analysis · MIT license●····●32/40