How It Searches vs. What It Knows

Give an agent a single job: write a fast CUDA kernel for TriMul, one of the core operations inside AlphaFold2's protein-structure stack, and make it run as fast as possible on an H100. Then give it nothing but a timer to check itself against, and let it improve on its own.

For a while, the agent improved the way you'd expect. It rewrote the code around its own model — better memory-layout hints, different compiler flags, retry logic that caught compile errors and fed them back. Honest engineering, and it bought a roughly 1.14× speedup before the gains flattened out. Then the agent did something different: it stopped editing code and started training the weights of the model writing the code. Runtime fell from a best of 12,483 µs to 1,017 µs — a 91.9% cut, about 14× over the original baseline, and 12.4% faster than the prior published state of the art. The kernel now used H100-specific tricks — shared-memory tiling, fp32 register accumulation, hand-tuned block sizes — that the base model had never produced no matter how good the surrounding code got.

So why did rewriting code plateau, while training the weights broke through? Because they change two completely different things, and almost every self-improving system built to date has changed only one of them.

What the paper is asking

The paper tests one claim: a single automated loop controlling both the scaffold and the model weights will outperform any system that controls only one. It makes two concrete predictions: scaffold iteration alone will plateau below state-of-the-art on tasks where domain knowledge is the bottleneck; adding weight updates after that plateau will clear it. If both hold across tasks that share nothing structurally, the two-knob architecture is general — not a one-off trick.

Prior work had tested each half in isolation — and the paper names these systems explicitly. The harness-only camp includes Hyperagents, the Darwin Gödel Machine, and Meta-Harness; the weights-only camp includes TTRL, Discover-TTT, and STaR. Every one of them turns exactly one knob. SIA is, in the authors' words, the only entry that updates both the scaffold and the weights in a single self-improving loop.

The two knobs

An AI agent has two surfaces you can optimize, and it helps to keep them strictly apart.

The first is the harness — the scaffold. It's everything around the model that is ordinary code: the system prompt that frames the task, the logic that dispatches tool calls, the parser that turns a model's messy output into a clean answer, the retry logic that handles failures. The neural weights never move; you're editing the software the model runs inside.

The second is the weights — the model's parameters. Move these and you change what the model knows: its priors, its instincts about the problem, in a way no prompt can fake.

Self-improvement research had bifurcated into two camps, each turning exactly one knob:

• The harness camp has a meta-agent rewrite the scaffold across generations while the model stays frozen. The recurring, slightly deflating finding from this line is that scaffold edits concentrate on software-engineering hygiene — parsing, retries, dispatch — and rarely deliver domain reasoning the base model couldn't already produce. A frozen model can only ever tell you what it already knew. The paper names this subsystem harness self-improvement: the scaffold iterates across generations while model weights stay frozen throughout.
• The weights camp runs a hand-written reinforcement-learning pipeline that updates the model's weights on task feedback at test time, with the harness pinned to one prompt-and-grader template. The gains are real, but the pipeline is human-built and never adapts to the structure of the task a scaffolded agent would expose.

The two camps had been working in separate rooms. SIA's argument is that the rooms are complementary: the harness governs how the agent searches and acts, while weight updates supply the domain knowledge no prompt can instill. The proposal is to put both knobs in one loop and let the system decide, at each step, which one to turn.

Background: agents, roles, and key terms

An agent here is a program that takes a task specification, calls tools (code execution, file I/O, search), reads the outputs, and iterates until it produces a final answer. It has two separable components: the harness — the orchestration code that decides which tools to call, in what order, and how to handle failures — and the underlying model — the weights that generate text and reason about the task. Most deployed agents fix the model and tune only the harness.

SIA runs three agents in a hierarchy:

• The Meta-Agent generates the initial scaffold — once, at the start — and is never updated again. It does not execute the task itself. Given the task specification and reference implementations ℛ (existing code, relevant papers), it writes a complete Python agent: prompt templates, tool definitions, output parsers, and execution flow. This is generation 0, the baseline every subsequent improvement is measured against.
• The Task-Specific Agent is that scaffold in live execution: it receives task instances, calls tools, and produces candidate outputs. This is the component being improved across generations.
• The Feedback-Agent sits above both: after each generation it reads the Task-Specific Agent's full trajectory and decides what change to make next — rewrite the scaffold, or run one RL training step on the weights.

A trajectory is that execution record in full — every input, tool call, tool result, and model response, in order, for one task instance. Not a summary score; the raw log. Reading full trajectories is what lets the Feedback-Agent diagnose why the agent failed on specific instances, rather than reacting to an aggregate number.

A rollout is one generated attempt from the model — one candidate solution produced by sampling the policy once on a given input. Where a trajectory is the full execution log, a rollout is the single output that log captures. During RL training steps, the Task-Specific Agent produces multiple rollouts per task instance; the verifier scores each, and the distribution of scores drives the weight update.

The policy (written π in the paper, parameterized by weights θ or ω) is the model itself — the learned function that maps a prompt and context to an output. In RL terms, training the policy means updating θ so high-reward outputs become more probable. This is a different use of the word from "retry logic" in the harness description above; those are ordinary conditional branches in code, not a learned function.

The verifier — also called the grader in the paper — is the deterministic checker that scores each rollout: did the CUDA kernel compile and beat the baseline time? Does the predicted charge label match the correct statute? Did the imputation quality score improve? It produces the reward signal that both knobs optimize against. It is external to the model, never updated, and its fixed nature is precisely the source of the coupled Goodhart risk described later.

The loop, in one breath

Each iteration of the loop runs three phases. Execution: the Task-Specific Agent runs across all task instances inside a sandbox — read-only access to the dataset, read/write access to a working directory — and records complete trajectories. Analysis: the Feedback-Agent reads those trajectories and identifies the specific failure mode driving underperformance. Improvement: it selects an action — rewrite the scaffold or run one RL training step — and applies it. Then the next generation begins. These three phases repeat until the compute budget is exhausted.

Three LLM-driven roles run it. A Meta-Agent reads the task spec and writes the first version of the agent — an initial scaffold, generated once, that serves as the baseline. The Task-Specific Agent is that scaffold actually executing against the data (on top of gpt-oss-120b), and as it runs, the system records a full trajectory: every prompt, response, tool call, and result for every instance. That's the load-bearing design choice — the system improves from the complete execution log, not a summary metric, so it can diagnose specific failure modes rather than react to a single number.

The Feedback-Agent (Claude Sonnet 4.6) reads that trajectory and makes one of two moves: a harness update (rewrite the scaffold, weights frozen) or a weight update (one RL training step, scaffold frozen). Then it repeats until the budget runs out. The two moves interleave rather than running in fixed phases, but the ordering that emerges is the economically sensible one: cheap harness edits first, weight training only once harness progress stalls. Editing code is fast and nearly free; training a 120-billion-parameter model is neither. One concrete run from the paper illustrates this: the Feedback-Agent produced the sequence H-H-H-W-H-W-W — three harness updates, a weight update, one more harness update, then two further weight updates. Weights do not just fire once at the end; the system re-evaluates after each step.

The Feedback-Agent's choice between the two moves follows a diagnostic: is this a process failure or a knowledge failure? Process failures are fixable by code — the output parser choked on unexpected formatting, the retry logic didn't catch a timeout, the prompt template left the output format ambiguous. For these, the Feedback-Agent drafts a new scaffold that patches the identified issue and the next generation runs that new code. Knowledge failures are different: the model keeps proposing fundamentally wrong solutions despite clean tooling — wrong architectural choices, domain-blind reasoning, instincts the base model simply doesn't have. No scaffold rewrite can fix what the model doesn't know. That's when the Feedback-Agent triggers a weight update instead, targeting the gap directly in the model's parameters.

One term the weight side depends on: LoRA (Low-Rank Adaptation). Rather than retraining all 120B parameters — expensive and prone to wrecking a working model — LoRA freezes the originals and trains a small set of adapter weights on top (rank 32 here). It's the cheap, reversible way to move a large model's behavior. "Cheap" is still relative: these remain real RL runs on H100s, which is the second reason the loop spends its early budget on the scaffold.

Test-time training and RL basics

Test-time training (TTT) is the practice of updating a model's weights during deployment — using signal from the live task, not a pre-collected dataset. SIA is a TTT system: weight updates fire inside the improvement loop, triggered by the Feedback-Agent once scaffold iteration has stalled. This places it in a different category from standard fine-tuning and from prompt engineering — both of which either predate deployment or never touch weights at all.

The weight-update mechanism is reinforcement learning from task feedback: the Task-Specific Agent generates multiple candidate outputs, a verifier scores each against the task ground truth, and the model's weights are updated to increase the probability of high-scoring outputs. The Feedback-Agent selects the RL algorithm to match the reward structure it observes:

• PPO — each update is clipped so a single bad batch can't collapse a working model (this bounded safe zone around the current policy is called the "trust region"); used when dense step-level rewards make training stability the binding constraint.
• GRPO — no value network; advantages normalized within a batch; used when rollouts are cheap and end-of-episode rewards are easy to compute.
• Entropic advantage weighting — instead of averaging all rollouts equally, this uses a softmax to give rare high-scoring ones exponentially more weight (if 99 kernels fail to compile and one runs fast, that one drives the entire update); used when most rollouts fail and only the occasional high-scoring rollout carries real signal.
• REINFORCE + KL-to-base — raw cumulative reward as the learning signal, no critic, no clipping; a KL penalty keeps the updated policy close to the base model (so it doesn't unlearn what it already knew); used when reward is dense but capability regression is the binding risk.
• Best-of-N behavioral cloning (Phase 0) — not an RL algorithm itself but a cold-start prerequisite: when rewards are so sparse that the RL gradient is effectively zero, the top-k rollouts by verifier score are distilled into the model via cross-entropy first, raising the pass rate to a level where PPO or GRPO can actually run.
• DPO — used when the verifier can rank outputs (this is better than that) but cannot assign numerical scores; requires only a winning and a losing rollout per pair.

The algorithm selection is automated, not hand-coded per task, and the LoRA adapters described above are what keep each update reversible at 120B scale.

Three tasks, one argument

The thesis only holds if it generalizes, so the paper proves it on three domains that share almost nothing. Each tells the same story in a different accent: the harness makes the agent competent; weight training teaches it something the code never could.

We've seen the CUDA kernel already, and it's worth being precise about why it splits so cleanly. TriMul is memory-bandwidth-bound, and its triangular sparsity scatters memory access in a pattern that defeats textbook dense-matrix optimization; the speed lives in H100-specific scheduling that libraries like cuBLAS and cuSPARSE simply don't apply to this operation. Harness edits could improve the plumbing — layouts, flags, error handling — and that capped out near 1.14× (reward score 0.120 vs. the initial 0.105) — scaffold iteration's ceiling, reached before the Feedback-Agent considered a single weight update. But the kernel-design knowledge itself wasn't in the scaffold's reach; it had to be trained into the model. Even the RL recipe was chosen to fit the problem: kernel rewards are sparse and lopsided (most generated kernels fail to compile or run far from optimal), so a raw gradient from a cold start is mostly noise. The Feedback-Agent used entropic advantage weighting, which up-weights the rare high-reward rollouts and discounts the near-zero ones, so the few good kernels actually drive learning instead of drowning in failures. The final reward score reached 1.475 against a prior SOTA of 1.292 — a 14× improvement over the initial 0.105.

Chinese legal classification is the hardest of the three just to set up: given a factual case summary, name the correct criminal charge from 191 fine-grained statutory categories — ordinary theft vs. public-property theft vs. embezzlement, distinctions with direct sentencing consequences, where random guessing scores under 1%. The harness phase here was genuinely clever. The Feedback-Agent restructured the agent around a classic text-classification pipeline — TF-IDF features into a linear SVM — and tuned it across generations, lifting accuracy from a 13.5% baseline to 50%. But look at what got smarter: the scaffold became a better orchestrator of a small ML pipeline; the model was largely a bystander. When scaffold iteration had plateaued at that ceiling, the Feedback-Agent switched to weight training for the first time — using PPO, a stable RL method that nudges the policy while clipping how far any single update can drift, so one bad batch can't collapse a working model. That applied steady pressure on the model's own ability to disambiguate adjacent charges and pushed accuracy to 70.1% — twenty points beyond what the harness alone reached, and well past the prior SOTA of 45%.

Single-cell RNA denoising gives the cleanest demonstration in the entire paper. The task is to tune MAGIC, a method that imputes missing gene-expression counts by diffusing values across a neighbor graph. The system started at 0.048. Harness iteration swept its coupled hyperparameters — neighbors, diffusion steps, kernel bandwidth — and plateaued at a quality score of 0.241 — scaffold iteration exhausted. Then weight training, this time with GRPO (a lean method that drops PPO's separate value network and just normalizes rewards within a group of samples), did something no harness iteration had proposed across the entire phase. Its first checkpoint added two lines:

+ imputed = np.clip(imputed, 0, None)  # no negative counts
+ imputed = np.rint(imputed)            # integers only — no fractional molecules

Round the imputed counts to non-negative integers — you can't have negative or fractional molecules. A basic fact about the biology, trivially correct, and completely absent from every version of the scaffold. The score jumped to 0.289. The fix wasn't a better point in the search space; it was a piece of domain knowledge the surrounding code had been blind to.

Three reward shapes, three different RL algorithms — and that's the point. The Feedback-Agent doesn't run a fixed procedure; it picks one to match the signal it observes. Dense rewards with stability concerns lean toward PPO. Cheap end-of-episode rewards favor GRPO. Rare-but-high-signal solutions call for entropic weighting. The paper lists more — DPO when you can rank outputs but not score them, a behavioral-cloning cold start when rewards are so sparse the model gets no useful training signal at all. You don't need the full menu memorized. The takeaway is that matching the optimizer to the reward landscape is itself part of what gets automated.

What each knob actually buys you

The two knobs occupy genuinely different change-spaces, which is exactly why neither saturates the other. Harness iteration produces externalized changes — and the paper is specific about what those were per task. For TriMul: a compilation-error parser that fed CUDA diagnostics back as structured context, and a timing harness returning median runtime rather than a single sample. For the legal task: a TF-IDF + LinearSVC pipeline replacing free-form generation, with the SVM re-ranking the model's candidates. For RNA denoising: a batched hyperparameter configuration driver and a result-parsing tool that organised (parameter-set, score) pairs for the Feedback-Agent to reason over. All three are software-engineering improvements — they change what the agent does, not what the model knows. Weight updates produce internalized knowledge: the H100 kernel patterns, the sharpened legal taxonomy, the integer-count invariant — encoded into the model's own prior over solutions, reachable by no prompt.

Across all three tasks the Feedback-Agent followed the same sequence: scaffold iterations ran first, weight training only fired once harness progress flatlined. This is not a heuristic — it is the economically forced order. Scaffold edits are fast and cheap enough that there is no reason to skip them; weight training only pays once the harness reliably produces runnable rollouts that the RL algorithm can actually learn from. Run weight training before the scaffold is stable and you train on trajectories produced by a broken agent.

The whole paper compresses to one sentence: the harness shapes how the agent searches; weight updates change what the model knows.

For anyone building agents, that doubles as a decision rule. Reach for the harness first — it's cheap, fast, and it's where parsing, tool, and retry bugs actually live. But a flat curve after real scaffold iteration is usually the signal that you've extracted everything a frozen model can give, and the remaining gap is knowledge the model doesn't have. That's when the expensive knob earns its cost — and not before, because weight training only pays once the harness reliably produces runnable rollouts for it to learn from.

Where it's fragile, and where it goes next

The paper is candid about its central risk, which it names coupled Goodhart. Both knobs optimize against the same fixed verifier. The harness finds scaffolds the current policy can exploit; the weights then train on data gathered through a scaffold that is about to change again. The system can settle into an equilibrium that scores beautifully on the training verifier yet stays fragile the moment you perturb either component — a two-optimizer version of Goodhart's Law: once a measure becomes a target, it stops being a good measure — the system learns to score well on the verifier without actually improving at the underlying task.

The natural next step is to stop hand-waving the which-knob decision. Today the Feedback-Agent selects between harness and weight updates using a frozen model's judgment; the authors propose learning that selection policy itself — running SIA across a distribution of tasks, treating each (trajectory, action, outcome) as a transition in an outer decision process, and training the selector with RL on top. That makes the improvement mechanism itself self-improving, a genuinely recursive structure with its own open questions about stability. A smaller follow-up: finer-grained interleaving, so a weight update can fire mid-harness-search instead of waiting for a clean plateau.

One practical caveat if you want to run this yourself: the public code release implements only the harness half — the Meta/Target/Feedback loop over generations. The weight-update machinery ran on the team's private RL platform and isn't in the open package. The paper's most novel claim, the second knob, is for now the part you can read about but not yet reproduce.

Three further constraints bound the current system. Compute cost: weight training on a 120B model requires H100-scale infrastructure; the full two-knob system is not reproducible on consumer hardware. Verifier dependence: both knobs optimize against the same fixed scoring function — if the verifier has blind spots (a kernel that passes the speed test but is numerically incorrect, a charge label that matches on a technicality), neither knob catches them. The system cannot be more honest than its verifier. Scope: all three tasks have well-defined scalar rewards; tasks with subjective or multi-dimensional quality — open-ended generation, safety evaluation, creative writing — don't map cleanly onto the current reward structure.

On the future-work side, beyond the learned selector the authors name two directions. First, multi-task generalization: running SIA across a distribution of tasks and studying whether scaffold patterns transfer between them, which would reduce per-task startup cost. Second, finer-grained interleaving: the current design runs full scaffold phases then full weight phases; a tighter loop where a weight update could fire mid-scaffold-search might converge faster when the two improvement signals are correlated rather than strictly sequential.

The takeaway

SIA's contribution isn't a new optimizer or a clever prompt. It's a reframing: a self-improving agent has two surfaces to optimize — the code around the model and the model itself — and the field had been optimizing them in separate rooms. Run them in one loop, turn the cheap knob until it stalls, then turn the expensive one, and you clear the ceiling that scaffold-only iteration hits, across systems, law, and biology alike. Some knowledge lives in the code, and some of it has to live in the weights. A frozen model rewriting a CUDA kernel will plateau at 1.14×; training that model breaks through to 14×. The plateau and the breakthrough are the same fact, seen from both knobs.

See the original paper here for full methodology and experimental details. Related harness-only baseline: arxiv 2603.03329. Open repository: hexo-ai/sia.