This is the complete 12-step roadmap through the seven pillars that actually decide whether an agent works: context, tools, memory, loops, graphs, harness, evals - each with the Claude workflow that proves it.
Frontier models went from 30% to over 80% on SWE-bench Verified in a single year. Coding agents got dramatically, measurably better.
Meanwhile only 17% of executives say they have fully adopted AI agents across their company.
> Follow my Substack to get fresh AI alpha:movez.substack.com
Agents fail because context bloats. Because loops never converge. Because nobody can measure whether last week’s change made anything better. The demo works on your machine. Production is a different animal.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
This is the 12-step roadmap through the seven pillars that separate the two - each with its own failure modes, its own Claude workflow, and its own way of quietly killing your agent if you skip it.
Here’s the reframe that makes the rest coherent. These are not seven separate skills. They are one skill in seven projections.
Bad context breaks the loop.
A loop without evals never converges on anything you can trust.
A graph without evals scales your error instead of your throughput.
Memory without a harness evaporates the moment a session ends.
Tools without context management drown the window before the work begins.
Which means there’s no useful way to learn them in isolation - but there is a dependency order, and it runs foundation first, execution second, reliability last. That’s how these twelve steps are arranged.
01. Context - read what actually loads
Karpathy’s framing is the one to hold: the model is the CPU, the context window is the RAM. Context engineering is the art of filling that RAM with exactly what the next step needs - no more.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
The number that reframes everything: in Claude Code, roughly 7,850 tokens load before you type a single character - system prompt, auto memory, skill descriptions, CLAUDE.md, environment info, MCP tool names.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
> Your actual prompt is around 45. Everyone optimizes the 45 and never opens the 7,850.
/context prints your real breakdown by category, tells you which memory files loaded, and flags which tool call ate the most tokens.
Two numbers to watch: memory files (heavy means an overweight CLAUDE.md) and free space. Pair it with /memory to see the exact files in play.
02. Context - cut it, then layer it
Anthropic deleted over 80% of Claude Code’s system prompt for the Claude 5 generation and measured no loss on their coding evals.
Most context isn’t wrong - it’s guidance written for a weaker model that now just costs tokens and forces Claude to reconcile contradictions before it can start.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
> Two rules make the cutting safe:
- Delete in blocks, not lines - one sentence sits inside the noise floor of your evals and tells you nothing.
And convert absolutes into principles: instead of “never write multi-line comments,” say write code that reads like the surrounding code - match its comment density, naming, and idiom.
The rule gives a fixed answer, the principle gives Claude a way to find the right one by reading your repo.
- What survives goes into a tree, not a scroll. Anthropic’s guidance is a hard number: keep the project CLAUDE.md under 200 lines, holding only gotchas Claude can’t infer.
Everything else becomes a skill (description loads at startup, body on invoke) or a path-scoped rule (loads only when a matching file is read).
# payments-api
Subscription billing and invoicing for the web app.
## Gotchas
// Non-obvious. Claude cannot infer these by reading the repo.
- All shared types live in `src/types.ts` — one monolithic file.
- `Money` is integer cents, never a float.
- Webhook retries must stay idempotent — provider replays for 72h.
- `db/legacy/` is frozen. Read it, never edit it.
## Deeper guides
- Verification → `.claude/skills/verify/`
- Releases → `.claude/skills/deploy/`
// NOT here: directory tree, framework, test runner, language
// version. Claude reads those from the repo itself.03. Tools & MCP - what the agent can reach
Tools are how an agent touches the world, and their descriptions are context - which makes this the hinge between what the agent can do and what it can afford to know about.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
The old advice was to teach tools with examples.
That inverted: with current models, examples constrain Claude to the exploration space they describe. Design expressive parameters instead.
A status enum of pending | in_progress | completed teaches an entire lifecycle without a single example - the type is the documentation.
Anthropic reached state of the art on SWE-bench Verified partly through precise refinements to tool descriptions, not model changes.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
And deferring beats dumping. Claude Code loads MCP tool names only - around 120 tokens - fetching schemas on demand through tool search.
Applying retrieval to tool descriptions rather than loading all of them improves selection accuracy roughly threefold.
Which makes the description the discovery surface: say when the tool applies, not just what it does. A tool Claude can’t find is a tool you didn’t ship.
@mcp.tool()
def find_stalled_shipments(hours: int = 24) -> list[dict]:
"""Shipments with no scan event in N hours.
Use when ops asks what is stuck, or before an escalation review."""
return query(STALLED_SQL, hours)
@mcp.tool()
def reroute(shipment_id: str, hub: str, reason: str) -> dict:
"""Reroute a shipment. Writes an audit row — compliance
requires a reason on every manual intervention."""
return post_with_audit(shipment_id, hub, reason)
# "Use when..." is the discovery surface for tool search.
# `reason: str` is required, so the audit trail cannot be skipped.
# The type enforces what a paragraph of instructions would only ask for.04. Memory - what survives the window
Every long task eventually exceeds one window.
What happens at that boundary is a design decision most people never make - they let automatic compaction guess, then wonder why the agent forgot a constraint stated an hour ago.
The mechanics are specific and worth memorizing. Project-root CLAUDE.md and auto memory are re-injected from disk.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
But rules scoped with paths: and nested CLAUDE.md files live in message history - they get summarized away and don’t return until a matching file is read again.
Invoked skill bodies come back, capped at 5k per skill and 25k total, oldest dropped first and truncated from the end - so anything critical belongs at the top of a SKILL.md.
The reliable move is the oldest one in computing: write it to a file.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
A plan file, a progress log, notes the agent rewrites as it goes. It persists without occupying the window and survives compaction because it lives on disk.
Compress deliberately too - /compact focus on the auth bug keeps what you chose, and /clear is badly underused when the next task doesn’t depend on the last twenty messages.
05. Loops - when to stop
An agent loop is act → observe → decide → repeat. The entire engineering problem lives in that last step: how does it know it’s finished?
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
Left to itself, a model answers badly in two directions - declaring victory on half-built work, or grinding forever on something that was done three iterations ago.
Neither is fixed by a better prompt, because both are structural. The stopping condition belongs in code that sits outside the model’s judgment. For bounded work that’s a test gate or a schema check.
For discovery of unknown size, the converging pattern is loop-until-dry: keep going until K consecutive rounds surface nothing new.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
One detail makes or breaks it, and almost everyone gets it wrong first time: dedupe against everything seen, not against confirmed results.
Otherwise rejected findings reappear every round, the loop never runs dry, and you’ve built a machine that pays forever to rediscover the same dead ends.
const seen = new Set(); const confirmed = []; let dry = 0;
while (dry < 2) { // stop after 2 empty rounds
const found = await runFinders();
const fresh = found.filter((b) => !seen.has(key(b)));
if (!fresh.length) { dry++; continue; }
dry = 0;
fresh.forEach((b) => seen.add(key(b)));
// ^ dedupe against SEEN, not against confirmed.
// Backwards, and this loop never terminates.
confirmed.push(...(await verify(fresh)));
}06. Loops - who checks the answer
A converging loop still converges on whatever the model believed. The fix is a verifier - something outside the model’s own judgment whose only job is to try to kill the finding.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
If it survives, it passes. If not, it never reaches the answer.
> Three patterns worth having in hand:
Adversarial verify: for each finding, spawn N independent skeptics prompted to refute it; keep it only if a majority survive.
Perspective-diverse verify: give each verifier a distinct lens - correctness, security, does-it-reproduce - because diversity catches failure modes that N identical checks never will.
Judge panel: generate N attempts from different angles, score with parallel judges, synthesize from the winner while grafting the best of the runners-up.
› For each finding, spawn three verifiers - correctness, security, reproducibility. Accept only what survives two of three.
● 6 findings → 18 verifier agents, running in parallel
✓ missing auth on /invoices/:id 3/3 — accepted ✓ race in webhook retry 2/3 — accepted
● "unsafe regex in validator" 0/3 — killed
● "N+1 query in dashboard" 1/3 — killed 2 of 6 findings were confident and wrong. The loop caught them, not your reviewer, and not your users.Note where this points. A verifier is an eval running inline - the same discipline as Layer III, applied per-result instead of per-release. Teams that build good verifiers find step 11 much easier, because they’ve already written down what “good” means.
07. Graphs - what runs in parallel
Most people write agents as a straight line - step one, step two, step three, each politely waiting for the last. Then they notice half those steps never needed to wait at all.
A node is a unit of work an edge means this output feeds that input. If no data crosses, there is no edge - and the wait is pure waste.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
The workhorse shape is the diamond: fan out to gather breadth, reduce with plain code, synthesize with one agent.
The reduce step deserves emphasis because it’s where money leaks - flattening and deduping is flatMap and a Set, not an agent.
Edges are free. Spend agents on judgment, not plumbing.
The ceiling is genuinely high. Claude Code’s dynamic workflows coordinate up to 1,000 parallel subagents in one run, and the orchestration costs zero model tokens because it’s a script, not a conversation.
› Run a workflow to audit every route under src/routes/ for missing
auth. One agent per route file, verify each finding before reporting. ●
Claude wrote an orchestration script · launching… ✓ Scope 1/1 ✓ Fan-
out 18/18 one agent per file ◯ Verify 11/18 3-vote skeptics per finding
○ Synthesize 0/1 your session stays responsive — the fleet runs in the
backgroundThat architecture is how a team ported ~960,000 lines of the Bun runtime from Zig to Rust in six days, with 99.8% of the test suite still passing.
08. Graphs - what the shape costs you
Topology isn’t cosmetic - it’s the biggest lever you have on both latency and spend, and two choices carry most of it.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
First, parallel() versus pipeline(). A parallel() barrier makes everything wait for the slowest node before the next stage starts.
A pipeline() streams each item through all stages independently - item A can be in stage 3 while B is still in stage 1.
Default to pipeline. Reach for a barrier only when a stage genuinely needs every prior result at once, like a cross-set dedupe. “It felt cleaner” is not a reason; barrier latency is real, measurable, wasted time.
Second, model tiering per node. Every subagent inherits your session model unless the script overrides it, so a big run bills entirely at your top tier by default.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
Bounded, repetitive nodes - extract this field, classify this ticket - belong on a cheaper model the merge node where judgment actually happens stays high.
A hundred cheap fan-out nodes feeding one top-tier synthesis costs a fraction of the same job run flat, at the same final quality.
09. Harness - surviving session death
Anthropic frames the problem perfectly: a long task is a software project staffed by engineers working in shifts, where each new engineer arrives with no memory of the previous shift.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
Compaction alone doesn’t fix it. Even a frontier model looping across context windows on “build a clone of claude.ai” falls short - and it fails in two specific, repeatable ways.
First, the agent tries to one-shot the app, runs out of context mid-implementation, and leaves the next session a half-built undocumented feature to reverse-engineer.
Second, later in a project, an agent looks around, sees real progress, and declares the job done.
Both are fixed before any coding starts, by an initializer agent that runs once and builds the environment: an init.sh that starts the dev server, a progress log, an initial git commit, and a feature list in JSON - over 200 entries for the claude.ai clone, every one marked failing.
{
"category": "functional",
"description": "New chat button creates a fresh conversation",
"steps": [
"Navigate to main interface",
"Click the 'New Chat' button",
"Verify a new conversation is created",
"Check that chat area shows welcome state"
],
"passes": false
}
// 200+ of these, all starting false. Agents may ONLY flip `passes`.
// "It is unacceptable to remove or edit tests" — deliberately strong
// wording, because a model that CAN edit the spec WILL edit the spec.One detail worth stealing outright: they chose JSON over Markdown for that list, because models are measurably less likely to inappropriately rewrite JSON.
10. Harness - one increment per session
With the environment scaffolded, each coding session gets a contract: get oriented, pick exactly one feature, verify it as a user would, leave the repo clean.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
“Clean” means what it means on a real team - no major bugs, orderly code, and a next engineer who can start work without cleaning up someone else’s mess.
The orientation ritual is mechanical and saves tokens every time: pwd, read the progress file, read the git log, read the feature list, run init.sh, and test that the basics still work before touching anything.
That last check matters more than it sounds - without it, an agent starting a new feature on top of a broken app just makes the breakage deeper.
And on marking things done: Claude’s tendency is to complete a change, run some unit tests, and call it finished without ever checking the feature end to end.
[Assistant] Getting my bearings on the current state. [Tool] bash —
pwd [Tool] read — claude-progress.txt [Tool] read — feature_list.json
[Tool] bash — git log --oneline -20 [Tool] bash — ./init.sh start the dev
server [Assistant] Verifying fundamentals before touching anything… ✓
core chat, theme switching, conversation loading all pass [Assistant]
Picking the highest-priority failing feature. no guessing. no
archaeology. the previous shift left notes.Give it real testing tools and require it to verify as a human user would - browser automation, actual clicks.
That single requirement dramatically improved performance in Anthropic’s experiments, catching bugs invisible from the code alone.
11. Evals - a number, not a vibe
The breaking point is always the same sentence: users report the agent feels worse after changes, and the team has no way to verify except guess-and-check.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
Without evals, debugging is reactive - wait for complaints, reproduce manually, fix, hope nothing else regressed. You can’t tell a real regression from noise.
Start smaller than you think. Teams delay because they imagine needing hundreds of tasks; 20–50 drawn from real failures is a great start, because early changes have large effect sizes.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
Pull them from what you already test manually, your bug tracker, your support queue. Write them so two domain experts would independently reach the same verdict - ambiguity in a task becomes noise in the metric.
And build balanced sets: test where a behavior should fire and where it shouldn’t, or you’ll optimize an agent that searches for everything.
Combine three grader types deliberately.
Code-based - fast, cheap, objective; use wherever possible.
Model-based - rubrics for nuance, calibrated against humans, ideally one isolated judge per dimension rather than one judge scoring everything.
Human - the gold standard, used sparingly to calibrate the others.
And grade what the agent produced, not the path it took: checking for an exact sequence of tool calls is brittle, because agents regularly find valid approaches you didn’t anticipate.
task:
id: "fix-auth-bypass_1"
desc: "Fix authentication bypass when password field is empty"
graders:
- type: deterministic_tests # fast, objective, reproducible
required: [test_empty_pw_rejected.py]
- type: llm_rubric # nuance tests can't capture
rubric: prompts/code_quality.md
- type: static_analysis
commands: [ruff, mypy, bandit]
- type: state_check # the OUTCOME, not the claim
expect: { security_logs: { event_type: "auth_blocked" } }
tracked_metrics:
- type: transcript
metrics: [n_turns, n_toolcalls, n_total_tokens]
# The agent says "fixed". The state_check decides whether it was.
# Grade outcomes, not announcements.12. Evals - keeping the number honest
A suite nobody reads is a number nobody should trust. You won’t know your graders work until you read transcripts from many trials - when a task fails, the transcript tells you whether the agent made a genuine mistake or your grader rejected a valid solution.
> 📷 
（原图为课程架构图，Twitter CDN 当前不可访问）
Failures should feel fair: obvious what went wrong and why.
Two traps make good agents look bad.
A 0% pass rate across many trials usually means a broken task, not an incapable agent. Opus 4.5 initially scored 42% on CORE-Bench - then a researcher found rigid grading that rejected “96.12” when expecting “96.124991…”, ambiguous specs, and irreproducible tasks. After the fixes: 95%.
The opposite trap is saturation - an eval at 100% tracks regressions but gives you no hill to climb, and real capability gains start showing up as noise.
Then hold two metrics apart. pass@k is the odds of at least one success in k tries - it rises with k. pass^k is the odds that all k succeed - it falls, fast. At 75% per trial, three trials all passing is only ~42%.
Use pass@k where one success is enough; use pass^k for anything customer-facing, where users expect it to work every time.
› Run the suite against the new model and compare to baseline. ● 47
tasks × 3 trials · 141 runs · parallel capability suite pass@1 61% → 74%
+13 regression suite pass@1 99% → 99% held consistency pass^3
52% → 68% +16 ● 2 regressions: refund_partial, escalation_tone →
transcripts written to ./eval-results/failures/ the upgrade decision
took an afternoon, not three weeksFinally, make it routine. Wire the suite into CI so it runs on every change and every model upgrade.
This is what turns a new model release from weeks of manual testing into a day of running your suite and reading the diff - and it’s the compounding advantage teams without evals never catch up on.
7 jobs to run with Claude - one per pillar
Open the window you never looked at. Measure first, then cut. A heavy memory-files number means an overweight CLAUDE.md - diagnosed in one command, fixed in an afternoon.
› /context then /doctor
Audit your tool descriptions. Every description should say when the tool applies, not just what it does. That sentence is the discovery surface - without it, a connected tool is one Claude never picks.
› Review every tool description in this MCP server. Add a "use when" sentence to each and tighten the parameter types.
Move state onto the filesystem. Have Claude keep a plan file it rewrites as it works. It survives compaction because it lives on disk -and long tasks stop losing the thread halfway through.
› Before you start, write the plan to plan.md and update it after each step. Re-read it whenever you resume.
Put a verifier on your loop. Pick any task where Claude decides for itself that it’s done. Add an external check and watch how often the first answer doesn’t survive three skeptics.
› After each fix, spawn three verifiers - correctness, security, reproducibility. Accept only what survives two of three.
Turn one linear task into a fan-out. Find a task where you loop over files or sources sequentially. One agent per item, running at once, then one merge. The wall-clock difference is the whole lesson.
› Run a workflow to audit every route under src/routes/. One agent per file, verify each finding, then synthesize.
Scaffold a multi-session project. Before a long build, have Claude write init.sh, a progress log, and a JSON feature list with everything marked failing.
› Act as an initializer agent: write init.sh, claude-progress.txt, and feature_list.json covering every requirement, all passes:false.
Turn last week’s bugs into a suite. Open your bug tracker and convert real failures into 20 tasks with unambiguous pass criteria. That suite is worth more than any framework you could adopt this quarter.
› Here are 20 real failures. Write each as an eval task with deterministic graders where possible and a rubric where not.
Conclusion:
Anyone can get a demo working. The job is everything after that.
The models keep getting better on their own schedule, and that improvement is free - it arrives whether or not you did anything.
What doesn’t arrive for free is the system around the model.
Context that stays lean. Tools an agent can actually pick between.
Memory that survives the window. Loops that converge and get checked.
Graphs that fan out instead of queueing. A harness that lets the next session continue the last one. And a number that tells you whether any of it got better.
Most people will keep waiting for a model good enough to not need any of this.
The ones who build the twelve steps will ship agents that work on days the model is having a bad one - which, in production, is the only reliability that has ever mattered.
个人点评
这篇是 Codez 的《AI Agent Engineer in 2026: 12 steps roadmap》——一份写给 2026 年 Agent 工程师的完整路线图。它最大的价值不是给你 12 个步骤，而是把 Agent 拆成 七个真正决定"能不能跑起来"的支柱，再给每个支柱配一条 Claude 工作流：context（上下文）、tools（工具）、memory（记忆）、loops（循环）、graphs（图谱）、harness（运行框架）、evals（评估）。
几个让我特别有共鸣的地方：
1. "七个支柱其实是一件事的七个投影"这个归纳很准。 很多人学 Agent 是一块一块学的：先学 context 管理，再学工具调用，然后补 evals。这篇把它收敛成一句话——坏 context 会破坏 loop，没 evals 的 loop 永远不会收敛。这个"失败模式联动"的视角，比单独学每个部分更能解释为什么 Agent 上线就崩。
2. "Production is a different animal"——demo 和生产的落差。 几乎所有 Agent 项目都在演示时完美、部署后崩溃。这篇把原因拆出来了：context 膨胀、loop 不收敛、session 死亡。这跟我自己做 Hermes、做多 Agent 编排时的体感完全一致——本地跑通只是起点，能稳定跑一周才是真考验。
3. 每个支柱都对应一个 Claude 工作流，这是全文最实操的部分。 比如 "Loops - who checks the answer" 讲的不是让 Agent 循环，而是谁在检查结果质量；"Harness - one increment per session" 讲每次会话只前进一个增量，避免 session 死亡时全丢。这些是真正能落地进工作流的判断。
4. "The agent says fixed, the state_check decides whether it was. Grade outcomes, not announcements." 这句话值得所有做 Agent 的人抄下来——评估看的是结果状态，不是 Agent 嘴上说了什么。这直接对应我记忆中"评估要能落地到当前实践"的原则。
两点补充：
- 这篇偏纯 Engineering 视角，把 Agent 当系统在讲。如果你更关心"Agent 如何服务内容生产/内容运营"这个场景，需要额外补充一层：评估指标（evals）不该只衡量代码正确率，还要衡量内容质量维度（信息密度、结构完整性、是否符合账号调性）。
- 12 步覆盖了七个支柱，但没有单独展开"多 Agent 协作"（multi-agent orchestration）的成本收益。实际工作中，引入第二个 Agent 是增加系统复杂度的决策，需要权衡。这是路线图之外值得你自己实验的部分。
总的来说，这是一份写给正在认真做 Agent 的人的路线图，不是概念科普。适合：正在搭 Agent 系统但遇到"本地能跑、线上就崩"的人，做多 Agent 编排的人，以及想给自己的 Agent 加一层可靠评估的工程师。建议把 12 步当检查清单，对照自己的 Hermes / Agent 项目逐条过一遍。
> 转载声明：本文原载于 X（Twitter），作者 Codez（@0xCodez）。原文链接：https://x.com/0xCodez/status/2089393338977829278
原文链接：https://x.com/0xCodez/status/2089393338977829278 | 作者：Codez（@0xCodez）
本文为转载文章，内容有删改。