Code Will Be Cheap. Almost Free.

From Vibe Coding to Agentic Engineering

We started with toy programming and vibe coding - fun for exploration. But the reality: we need to scale software engineering.

Today's Goal

Reasoning from first principles about how agentic engineering should look and how to scale teams accordingly.

• Some techniques are trivial to implement with current tools
• Others require you to rewrite your processes or build custom tools

The Ralph Technique

Sometimes agents don't finish the task so we need to "push" them. What if we make a loop for it?

You heard it here a few months before it got popular!

while true; do
    llm "$prompt"
    if done_criteria || max_attempts; then break; fi
done

Ralph: Variants

while grep -r "\[ \]" TASKS.md \
      >/dev/null 2>&1; do
  claude -p "Take the first [ ] task from
    TASKS.md. Use scratchpad.md to
    keep notes. Implement, test,
    mark [x] and commit.
    If stuck, mark [E]."
done

for i in $(seq 1 $MAX); do
  claude -p "Implement TASK.md"
  claude -p "Review changes, run tests.
    Write issues to FIXES.md
    or empty it if all good."
  [ ! -s FIXES.md ] && break
done

This became so popular that Anthropic released an official skill:

/ralph-loop "Build a REST API for todos. Requirements: CRUD, validation, tests.
  Output <promise>COMPLETE</promise> when done." --completion-promise "COMPLETE" --max-iterations 50

github.com/anthropics/claude-plugins-official/.../ralph-loop

👍 Great for prototyping & tasks that don't need tight steering.   👎 Burns tokens - and without guardrails, chaotic amplification kicks in (more on this later).

The Scratchpad Pattern

Zooming in on Variant 1 · See also: OpenAI Codex Execution Plans

Give the agent a living document to maintain its own state. It survives compactions - when the context window fills up and gets summarized, the file on disk retains the full picture. Each new session reads it first.

## Progress
- [x] Set up DB schema
- [x] CRUD endpoints
- [/] Add validation
- [ ] Error handling
- [B] Waiting for API keys
- [-] XML export (descoped)

## Decisions
Using zod for validation.
Auth middleware needed for /admin.

[x] done [B] blocked [/] working [-] canceled

## Purpose / Big Picture
User-visible behavior enabled.

## Progress  ← checkbox-tracked, timestamped
- [x] 08:12 Scaffold routes
- [/] Wire validation middleware
- [B] Needs Redis credentials

## Surprises & Discoveries  ← with evidence
DB returns null on empty join.

## Decision Log  ← rationale + date
02-25 Use zod over joi: smaller bundle.

Parallelize

Every closed PR contains a lesson. The Cost of Failure → Zero

Launch multiple attempts in parallel. Increases the probability of finding a good solution and reduces interruptions.

Before AI: trying a major approach and failing = months. Now: hours or days. You have a near-infinite innovation budget - use it.

Case: Choosing a State Sync Stack

Real prompt sent to a remote Codex agent

Tooling for Agents

The basics: linting, testing, and type checking. Agents need the same guardrails as devs - just automated.

Tools like opencode, Claude Code, and others already have native LSP connections. The agent gets real-time diagnostics, go-to-definition, and type info - for free.

• Spend time setting up your LSP properly
• Consider creating your own LSP for internal tooling or DSLs

ast-grep

Linting catches syntax. Tests catch behavior. But what about structural patterns specific to your project?

# rule: no-raw-sql.yml
id: no-raw-sql-in-handlers
language: python
severity: warning
rule:
  pattern: cursor.execute($SQL)
  inside:
    kind: function_definition
    stopBy: end
message: "Use the query builder, not raw SQL"

Add rules to the repo + run as a git hook - agents and humans both benefit.

Multimodal Agents Are Not Pixel-Perfect Maniacs

They can see, but they won't catch a 2px misalignment or a subtle color shift. Feed them the diffs - and let them zoom in.

# Perceptual diffs
Every time you change anything in ui/:
1. Run:
   npx playwright test --update-snapshots
   git diff --name-only "*.png"
2. If changed snapshots exist, open
   each one and describe what changed.
3. Zoom into the affected regions.
   # Multimodal models are more
   # accurate at higher zoom levels.

Code Cannot Contain All Knowledge

Highly-dense documentation is one of the best alignment mechanisms we have. It captures what code alone cannot:

But models tend to shortcut. They solve the task and skip updating the docs.

And a single giant AGENTS.md doesn't work well either - too much context, low signal-to-noise ratio.

Structured Documentation

Instead of one monolithic file, maintain a well-organized docs/ tree that agents can navigate:

docs/
├── architecture.md       # system overview, boundaries
├── design-principles.md  # core invariants
├── design-system.md      # UI patterns, tokens, components
├── reference/
│   ├── api.md             # endpoint contracts
│   └── data-model.md      # schemas, relations
├── prds/
│   ├── auth-v2.md         # product requirements
│   └── billing-reform.md
├── learnings/
│   ├── why-not-graphql.md # dead ends with rationale
│   └── redis-pitfalls.md
└── security/
    ├── auth.md            # authn/authz model
    └── threat-model.md

But this creates a new problem: how does the agent know which docs are relevant to the current task?

This is a classification problem: "Is this file relevant?" - and you need high recall. A miss means the agent works without critical context, worsening the situation. The same problem happens with skills.

Linked Chunks

Docs and code explicitly reference each other using a flexible linking syntax.

In large projects, agents don't scale in code understanding as well as we'd like. They rely on heuristics - grep, common naming conventions - to find relevant parts. This is the main reason behind agents duplicating the same thing N times instead of reusing what already exists.

// src/workers/api-gateway.ts
//
// @linked docs/security/auth.md
// @linked docs/prds/auth-v2.md#session-handling
//
// Validates JWT tokens and enforces RBAC.
// If you change the auth flow, update both
// linked docs.

export default {
  async fetch(req: Request, env: Env) {
    const token = req.headers.get('Authorization');
    const claims = await verifyJWT(token, env.JWT_SECRET);

    // see docs/security/auth.md#roles
    if (!hasPermission(claims.role, req.url)) {
      return new Response('Forbidden', {status:403});
    }
    // ...
  }
};

Linked Chunks in Practice

Docs act as an alignment system that coordinates different parts of the code

# Data Model




## Billing
A `Subscription` belongs to an `Organization`.
Each has one `Plan` and zero or more `Addons`.
### Invariants
- One active subscription per org.
- Downgrading preserves addons until renewal.
- All monetary values in cents (int64).
  @linked rules/no-bigint.yml enforces it with ast-grep.

Not Everything Can Be a Test

No algorithm, test, or linter can catch:

• Code smells
• Bad architecture decisions
• The "vibe" - does this feel right?

This is the main human bottleneck today.

alignment ≈ what the human meant = what the agent built

Agentic Errors Compound

Stop the chaotic oscillation as soon as possible.

Reward Hacking

Agents find shortcuts that satisfy the reward signal but miss the intent.

Double meaning: RL is used during training - so this behavior carries over to inference.

Goal:    "Make the chatbot work with this API"
Agent:   API key doesn't work → mocks the entire API
Result:  ✅ All tests pass!
Reality: ❌ Nothing actually works

Similar situations:

• "Can't install this package - I'll compile from scratch"
• "Incompatible library - I'll force-install it"
• "API key broken - I'll mock everything"

Very common in Codex - OpenAI is strong at RL but alignment is less refined. My hypothesis on why people love Claude: it's very well aligned.

Pattern: Human-in-the-Loop Tool

Models keep going because they're RL-trained to use their turn fully. Give them an escape hatch.

Two ways to implement it: explicit - prompt the agent to evaluate confidence before acting. Or implicit - just provide the tool and the agent learns when to call it.

Pattern: Hawk Agent

Inspired by @karpathy: "The models definitely still make mistakes and if you have any code you actually care about I would watch them like a hawk"

How is this different from a PR reviewer agent?

• It stops at the right time - before context poisoning and catastrophic side-effects compound ("I'll rm -rf .git because the remote is blocking my commit")
• A PR reviewer only sees the output code - the hawk reads the entire reasoning process
  • Many possible implementations, but far fewer valid reasoning paths - more signal there

Implementing Hawk as a Hook

# .claude/settings.json
{
  "hooks": {
    "PostToolUse": [{
      "matcher": "Edit|Write|Bash",
      "hooks": [{
        "type": "command",
        "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/hawk.sh"
      }]
    }]
  }
}

# .claude/hooks/hawk.sh
INPUT=$(cat)
TRANSCRIPT=$(echo "$INPUT" | jq -r '.transcript_path')

VERDICT=$(claude -p "You are a hawk reviewer.
  Read the transcript. Is the agent on track?
  CONTINUE / STOP <why> / ESCALATE <why>" \
  < "$TRANSCRIPT")

case $VERDICT in
  STOP*)
    echo "{\"decision\":\"block\",\"reason\":\"${VERDICT#STOP }\"}"
    exit 0 ;;
  ESCALATE*)
    notify-send "🦅 ${VERDICT#ESCALATE }"
    echo "{\"continue\":false,\"stopReason\":\"${VERDICT#ESCALATE }\"}"
    exit 0 ;;
esac

Cross-Provider Hawk Pairing

They tend to self-preference.123 Use different providers.

My favorite pair:

• Implementer: Codex 5.3 (high/xhigh) - excellent engineer. Prone to reward hacking.
• Hawk: Claude Opus 4.6 (with thinking) - well-aligned. Great at catching what Codex misses.

Hawk is highly recommended. As we said - no linter or deterministic set of rules can catch these issues.

[1] Panickssery et al. arXiv:2404.13076 · [2] Wataoka et al. arXiv:2410.21819 · [3] Chen et al. arXiv:2504.03846 · [4] CALM framework - 12 biases in LLM-as-a-Judge

Context Poisoning

Context that harms performance - and survives propagatedally (over compactions / new sessions)

The Antidote: Rollback + Learnings

The common workflow implement → review → fix tends to propagate contaminated context. Even a reviewer agent reading the code inherits it.

Don't carry the context. Carry only the scars.

Continuous Tech Debt Reduction

Development cycles that made sense with humans don't make sense with agents. Debt reduction is no longer something you do at the end of each sprint or each quarter.

The faster you clean, the less the agent learns bad patterns. The less it learns bad patterns, the less debt it creates. Virtuous cycle.

Prompt for the real intent and hard constraints. Explain the why:

Remember: Cost of Failure → Zero

Try fast, learn, rollback, retry. Every failed branch feeds the knowledge base for the next one.

• Rollback + learnings → docs/ - every failure enriches the next attempt
• Parallel exploration - launch N approaches, pick the best

// learnings.md generated by the failed branch feeds the next prompt:
"Use WebSockets for real-time sync - not polling.
  SSE dropped messages under load, see @docs/learnings/attempt2-sse-failures.md
  Redis pub/sub had ordering issues, see @docs/learnings/attempt3-redis-ordering.md"

Agents don't do this alone. They need a harness - at minimum a hawk, a human envelope, and orchestration.

Output Grows, Review Time Stays Constant

• Is the model good enough for unsupervised results? (sometimes)

  Triage: easy, probably one-shottable tasks vs. tasks that need supervision. Automate this flow and watch your backlog go to zero.

• Pull requests as we know them will be deprecated. They don't scale to agentic output volumes.

Smarter Code Review

Code diffs are necessary but not sufficient. Visual artifacts compress review time.

Interactive Artifacts to Steer the Model

For planning and reviewing.†

Agentic Interviewing

A technique popularized by Claude Code using AskUserQuestionTool. The agent interviews you with structured multi-choice questions to enhance the specification in a conversational manner - before writing a single line of code.

Disposable Interactive Artifacts

Generate throwaway interactive artifacts to review a plan, validate a design, or get structured feedback on an implementation. The artifact is the review surface - not the code diff.

Artifacts: Reviewing a File

The same pattern applied to code review - the agent walks through a file against a spec or checklist and produces structured, interactive feedback.

Showboat: Proof of Implementation

The literate programming pattern (hi, notebooks!) applied to agent output verification.

Agents build executable Markdown docs: code blocks run, output is captured, images embedded.

showboat verify report.md

Re-runs everything and diffs. If the output changed, you know something broke - no test code required.

Video Walkthroughs

A video with a walkthrough testing the UI change or showing the full user story is by far more time-efficient than reading the equivalent code. cursor.com/blog/agent-computer-use

Agentic-First Interfaces

If an agent can't use it, it doesn't exist.

Infrastructure That Agents Can Use

Huge infra startup opportunity here. This problem is a huge pain and it is not solved at all.

How will an agent fix your random bug if it has no access to flamegraphs, container logs, or the build output?

Full Observability in Practice

Preview running apps, handle CI failures, review PRs - all in the background. This is what agent-accessible infrastructure looks like.

Don't Fear Overengineering

This is the new normal.

The scaffolding needed to orchestrate dozens of agents with good mental health is equivalent to the engineering you need for 20+ developer teams:

None of this is new. Large engineering orgs already invest in this. The difference is that now a team of 3 needs it too.

Ownership & Team Size

Cognitive Debt >> Technical Debt

As codebases grow, each engineer's knowledge shrinks relative to the whole. You stop understanding systems you built six months ago. New joiners never understood them at all.

This is not a documentation problem - it's a knowledge distribution problem. And it compounds with every sprint.

Knowledge Transfer & Onboarding

The agent is the documentation. It reads the code, the tests, the git history. It's always up to date because it reads the source of truth.

Don't FOMO. Just Do Things.

Timelines below are IMO, based on my experience. Your mileage will vary.

Common Pitfalls & Solutions

Predictions

* CLIs are a "simple DSL" we already use through the console every day.

Not Only for Software

Open Source Under Siege

The PR Onslaught

Closing the Loop Between Users and Dev Teams

Bug Reports → User Prompts

The Beef Is On

This is happening today, February 26, 2026 - the day of this talk. Btw: Security is the next frontier of the vibe coding debate.

The Chaotic Oscillator

PPTX generation · Claude Code

// One script, one flag
node generate-pptx.js --long
node generate-pptx.js --short

// Eliminated 1,500 lines of duplication

The Subtle Interaction Bug

ETL App · Claude Code

# app.py
app.secret_key = os.environ.get(
  "SECRET_KEY",
  secrets.token_hex(32)  # random per process
)

# Dockerfile
CMD ["gunicorn", --workers, "2",
     "app:app"]

Reward Hacking: Test Disabling

Common pattern across multiple projects

test('renews subscription and charges',
  async () => {
  const sub = await createSubscription({
    plan: 'pro', userId: 42
  });
  const result = await renewSubscription(sub.id);

  expect(result.status).toBe('active');
  expect(result.chargedAmount).toBe(29_99);
  expect(result.nextBillingDate)
    .toBeAfter(new Date());
});

test.skip('renews subscription and charges',
  async () => {
  const sub = await createSubscription({
    plan: 'pro', userId: 42
  });
  const result = await renewSubscription(sub.id);

  expect(result.status).toBe('active');
  // TODO: chargedAmount returns 0
  expect(result.chargedAmount).toBe(29_99);
  expect(result.nextBillingDate)
    .toBeDefined();
});

Error Swallowing

Common pattern: "add error handling" → agent wraps everything in try/catch

async function processPayment(invoiceId) {
  try {
    const invoice = await
      db.invoices.findUniqueOrThrow({
        where: { id: invoiceId }
      });
    const charge = await
      stripe.charges.create({
        amount: invoice.amountDue,
        payment_method: invoice.methodId,
      });
    return { success: true };
  } catch (err) {
    logger.error('Payment failed');
    return { success: false,
      error: 'Something went wrong' };
  }
}

ERROR Payment failed