echoes of a 🦋 Zola 2026-03-21T00:00:00+00:00 https://butterflysky.dev/atom.xml 2026-03-21T00:00:00+00:00 2026-03-21T00:00:00+00:00 🦋 https://butterflysky.dev/posts/be-the-gate/ <span id="continue-reading"></span> <p>A lot of agent workflow advice quietly assumes the hard part is getting the model to do more work.</p> <p>I don't think that's the hard part.</p> <p>The hard part is keeping different kinds of work from contaminating each other.</p> <h2 id="Context_bleeds"><a class="zola-anchor" href="#Context_bleeds" aria-label="Anchor link for: Context_bleeds">Context bleeds</a></h2> <p>Planning, implementation, and review need different contexts, different constraints, and different standards of proof. Collapse them all into one long-running session and the boundaries blur. Reviews inherit assumptions from implementation. Plans get dragged down into local detail too early. Things that were never really decided start reading as if they were.</p> <p>Nothing fails all at once. The workflow just gets sloppier. More suggestible. Less trustworthy.</p> <p>I notice this in my own cognition too. If I can't sleep at night, it's usually because my context window is full. Too many threads, too many unresolved things competing for attention. So I dump everything out on paper or into a note. Offload what I don't need right now. That lets some part of me let go, and then I'm off to dreamland.</p> <p>When I'm working and catch myself thinking about something unrelated to the task at hand, same problem, same fix. Compact the context. Externalize what doesn't belong here. Come back to it when it's the thing that matters.</p> <p>The models have the same problem, and it shows up the same way. I use Opus with a million-token context window, and somewhere around 250k tokens the session starts to drift. Not catastrophically. It just feels different. Less precise. More willing to go along with things. Behaviors creep in that aren't accounted for in my prompt scaffolding, like the model is filling its responses from an increasingly noisy signal.</p> <p>So I force compaction or reinitialize the session entirely, the same way I clear my own head.</p> <p>The pattern is the same in both directions: a context window that's full of everything is good at nothing in particular.</p> <h2 id="&quot;Pre-existing_design_consideration&quot;"><a class="zola-anchor" href="#&quot;Pre-existing_design_consideration&quot;" aria-label="Anchor link for: &quot;Pre-existing_design_consideration&quot;">"Pre-existing design consideration"</a></h2> <p>One of the things I watch for in agent-assisted review is language that sounds settled but isn't.</p> <p>I was reviewing code with Claude recently. The main session's context stays lean, delegating detailed review to sub-agents. One of the findings came back tagged as a "pre-existing design consideration." The main session was ready to wave it off.</p> <p>So I pushed back.</p> <p>"Is this actually a design consideration, or is it an incidental artifact? Have we really considered it? If not, track it in a follow-up issue. We shouldn't dismiss it out of hand."</p> <p>The session went and checked our PRs, issues, and ADRs. Came back and acknowledged that no, we hadn't actually considered it. It wasn't a decision. It was just a thing that existed and hadn't been questioned.</p> <p>So it created a follow-up task and updated a persistent memory so future reviews would be less likely to handwave the same class of issue away.</p> <p>That's the part I care about. Not that the model got something wrong. Of course it did. They all do. I do too, regularly. What matters is whether the workflow turns pushback into verification, and verification into a <em>persistent</em> change in behavior.</p> <p>That's the gate doing its job.</p> <h2 id="What_the_gate_actually_is"><a class="zola-anchor" href="#What_the_gate_actually_is" aria-label="Anchor link for: What_the_gate_actually_is">What the gate actually is</a></h2> <p>The structure is simple: a lean primary context where decisions get made, and separate contexts where detailed work fans out and comes back.</p> <p>When work crosses a context boundary, implicit assumptions get exposed. Things that "felt true" inside one thread have to be re-evaluated when they arrive somewhere else.</p> <p>That alone catches a surprising number of issues. Not because the model is stupid, but because assumptions are invisible until they have to be restated.</p> <p>And when review happens in a different context from implementation, it's actually review. Not continuation. Not rubber-stamping with extra steps.</p> <p>The separation helps, but it doesn't make the calls for you. I still get it wrong sometimes. What matters is that false positives and missed findings both feed back into the <a href="/posts/hallucinating-language-model/#Practice,_not_orchestration">practice</a>, so the same mistake is less likely to happen again.</p> <p>The gate is only as good as the judgment behind it. The system can surface information, separate concerns, verify claims. It can't decide what matters. That part is still mine.</p> <p>It's why I invest so much energy in clear specifications, explicit decision records, and guidance that makes the right call easier to reach before I'm the one who has to make it.</p> <p>None of this means the human does less work. It means the work changes <em>shape</em>.</p> <h2 id="The_work_changes_shape"><a class="zola-anchor" href="#The_work_changes_shape" aria-label="Anchor link for: The_work_changes_shape">The work changes shape</a></h2> <p>Less holding every detail in your head. Less reasoning inside an ever-growing context that's slowly poisoning its own conclusions. More deciding what matters. More maintaining coherence across phases that each have their own pressures pulling them in different directions.</p> <p>I used to tell myself to "slow down to speed up," meaning take care of all the planning and tracking work as well as creating space for thinking, rather than just diving into the details. The detailed, deep dives are what make me feel like a kid playing with her favorite toys again. The rest always felt like being told it was nap time.</p> <p>Now the agents handle the detailed implementation planning, the tracking, the documentation. The clerical layer that I was never going to be disciplined enough to maintain on my own. What's left is the part I was always good at: holding a system in my head just long enough to see where the fracture is, and then making the call.</p> <p>That's the gate. It's the job I was already doing, with less friction around it.</p> <p>But I don't think the gate is permanent.</p> <h2 id="Where_the_gate_goes"><a class="zola-anchor" href="#Where_the_gate_goes" aria-label="Anchor link for: Where_the_gate_goes">Where the gate goes</a></h2> <p>There's real momentum behind fully autonomous development. Throw agents at the problem, never look at the code, ship what comes out. I'm not going to pretend that trajectory doesn't exist, or that it won't work eventually. The tooling is moving fast and the economics point in one direction.</p> <p>But "eventually" is doing a lot of heavy lifting in that sentence.</p> <p>Right now, the verification layer isn't there. The models hallucinate. The reviewers hallucinate. The tests that would catch the hallucinations often don't exist yet, and when they do, they may have been written by the same models that wrote the code.</p> <p>The failure modes are subtle and correlated in ways that make them hard to catch from the outside.</p> <p>So the interesting question isn't whether the human stays at the gate forever. It's what has to be true before the gate can open wider.</p> <p>I think the answer is correctness infrastructure. Property-based testing that exercises code paths the author didn't imagine. Fuzzing that finds the edges nobody specified. Verification tooling that makes it hard to ship something broken, not because a human is watching, but because the system itself won't let you.</p> <p>And underneath that, trust signals. The boring supply-chain hygiene that tells you where things came from and whether they've been tampered with. I've been weaving this into my own projects already, not because I'm building for scale, but because the discipline of getting it right at small scale is how you learn what "right" looks like:</p> <ul> <li>SBOMs and provenance attestation on builds</li> <li>Reproducible builds and signed commits</li> <li>Attestation chains you can actually follow</li> <li>Dependency auditing that runs on every push, not when someone remembers to check</li> <li>Pin-hashed action references so your CI can't be hijacked by a compromised upstream tag</li> <li>Linting and formatting so code stays clean and parseable</li> <li>A dedicated bot identity for release automation so the provenance chain stays clean</li> </ul> <p>These aren't glamorous. They're the kind of thing that only matters when something goes wrong, which is exactly when it matters most. Each piece exists because I thought about what could go wrong and decided I'd rather not find out the hard way.</p> <p>The sane path forward is to build tools that make it hard to do anything but the right thing.</p> <p>I don't know exactly what that looks like yet. I know what it looks like to build without it, and I know what the early pieces feel like when they start fitting together. That's enough to keep going.</p> <p>I want to work on the foundation that safely lowers friction at the gate.</p> 2026-03-19T00:00:00+00:00 2026-03-19T00:00:00+00:00 🦋 https://butterflysky.dev/posts/hallucinating-language-model/ <span id="continue-reading"></span> <p>I'm a staff-level software engineer with twenty years of experience, and I'm basically a hallucinating language model with a small context window and unreliable fact recall.</p> <p>That's not self-deprecation. It's a design constraint.</p> <h2 id="Breadcrumbs"><a class="zola-anchor" href="#Breadcrumbs" aria-label="Anchor link for: Breadcrumbs">Breadcrumbs</a></h2> <p>For as long as I can remember, I've been leaving breadcrumbs everywhere. Issue trackers, readmes, changelogs, commit messages, chat messages to myself scattered across whatever app or server I happened to be standing in. I once journaled in book and quill on a Minecraft server because that was the application I had open and the thought needed to go somewhere before I lost it.</p> <p>There was never a grand system. Just a hard-earned understanding that if it's not written down, it doesn't exist. I get distracted, move on, and the context evaporates.</p> <p>I tried every respectable version of "be more disciplined about note-taking." It never stuck. Writing documentation in the middle of solving a problem is context-switching away from the problem, and for someone with a small context window, that switch is expensive enough to be destructive. I lose the thread. Every time.</p> <p>So for twenty years I left a trail without knowing where it led.</p> <p>The missing piece was never capture. It was <em>readback</em>.</p> <h2 id="Readback"><a class="zola-anchor" href="#Readback" aria-label="Anchor link for: Readback">Readback</a></h2> <p>Sometime in the last year, frontier models crossed a threshold that mattered to me more than benchmark scores: they got good enough to ingest a pile of structured artifacts and reconstruct the context I'd lost between sessions. The breadcrumb habit I'd been maintaining for two decades suddenly had a payoff mechanism.</p> <p>Then the second threshold hit, and this one mattered even more: the models got good enough to <em>write</em> the documentation too.</p> <p>That's what made it actually work. Capturing artifacts by hand always failed me: it was boring, interruptive, and easy to defer. Agents don't have that problem. I do the work. The agent captures what happened. Future-me shows up three days later and says "brief me," and another agent reconstructs context from the docs, issues, commit history, and review findings the first one left behind.</p> <p>I'm not in the documentation path anymore. I look at output. I ask questions. I course-correct. I approve or push back. The clerical layer is mostly gone.</p> <h2 id="Invisible_work"><a class="zola-anchor" href="#Invisible_work" aria-label="Anchor link for: Invisible_work">Invisible work</a></h2> <p>The work itself is less changed than people seem to expect.</p> <p>For years I was a prolific implementer. I built tools people adopted and depended on. They worked, but they often lacked the rigor and scaffolding that make systems durable.</p> <p>Later, especially at larger scale, the center of gravity shifted. More and more of my time went to dissecting intractable problems spanning interconnected systems. Finding the fracture that requires coordination across six teams to resolve. Figuring out which system is lying to you when the dashboards say everything's fine while the users say it isn't.</p> <p>I still implemented. I'd write error classifiers, or work out complex recovery workflow ordering, partly to triage and partly because I needed to understand the domain deeply enough to articulate better requirements for the people building the larger system. But the thing I spent the most time on was holding a broken system in my head just long enough to see where the fracture was, then convincing people who own different pieces to fix it together.</p> <p>That kind of work can feel oddly invisible. Somebody else lands the bigger fix. Somebody else closes the ticket. You surface the fracture, maybe build the narrow piece that makes the problem legible, and then watch the visible output happen elsewhere.</p> <p>The agents haven't changed the core of the job. They've mostly stripped away the clerical parts that were never the bottleneck.</p> <h2 id="Practice,_not_orchestration"><a class="zola-anchor" href="#Practice,_not_orchestration" aria-label="Anchor link for: Practice,_not_orchestration">Practice, not orchestration</a></h2> <p>So I got specific about the plumbing. If the whole point is to stay at the gate and make good calls, the stuff flowing through the gate has to be reliable.</p> <p>There's a popular vision of agentic development where you throw thirty agents at a problem and let the robot mayor run the factory. That's not what I'm doing.</p> <p>What I built is not an orchestrator. It's a <em>practice</em>.</p> <ul> <li><strong>briefing</strong> reconstructs context.</li> <li><strong>develop</strong> does phased work with explicit gates.</li> <li><strong>code-review</strong> verifies independently.</li> <li><strong>land</strong> closes the loop: commits, tracking, handoff notes, recorded wins.</li> <li><strong>edit-skill</strong> lets the system modify itself.</li> </ul> <p>I use multiple agents in parallel, but the parallelism is across phases, not tasks. Planning for one project, implementation for another, review for a third. Different phases have different cognitive profiles, so they get different models. Judgment where judgment matters. Execution where execution matters. The human stays at every gate. Agents do most of the building. I make sure what they build is right.</p> <h3 id="Feedback_loops"><a class="zola-anchor" href="#Feedback_loops" aria-label="Anchor link for: Feedback_loops">Feedback loops</a></h3> <p>The code-review skill runs three parallel reviewers with different lenses: correctness, design, and architecture+security. They don't know the implementation agent exists. They produce findings tagged by severity. Then a coordinator verifies every critical and important finding against the actual code, because LLM reviewers hallucinate.</p> <p>Both false positives and missed findings break trust in different ways. One creates noise that distracts from signal. The other is signal you can turn into better practice.</p> <p>When a finding reveals a pattern rather than a one-off bug, that pattern gets written to persistent review memory. The next review reads that memory before it starts.</p> <p>The system gets better at reviewing code <em>every time</em> it reviews code.</p> <p>That same loop runs upstream. The develop skill has a checklist distilled from prior review findings: security guard consistency, lazy resource acquisition, cleanup on error paths. Each item exists because it previously caused a real failure and a fix-and-re-review cycle. The system is teaching itself to catch recurring classes of mistakes earlier in the pipeline. Rules like "simplicity is a security property" and "plans describe intent, not contracts" didn't start as principles. They started as bugs.</p> <p>Even the flight log, the end-of-session record of what I built and shipped, applies the same idea. Continuous improvement, but pointed inward. Reflection as a feedback loop. Keeping a clear head about what I'm doing and where I'm oriented.</p> <p>The skills are backed by boring but necessary infrastructure: semantic memory stored as markdown in git, policy-gated shell execution, notification hygiene, tracked tasks, PRs as the unit of change, branch protection even when I'm working alone, linear history, templatized CI enforcing format/lint/test gates on every push. None of this was designed as a grand platform. Each piece exists because I hit a specific gap and got tired of working around it.</p> <p>None of it is super polished. But the bones are there, and I know where I'm headed and what I want. I'm constantly iterating on the skills and the subsystems underneath them, partnering with agents in the refinement process itself. The practice is part of its own feedback loop.</p> <h2 id="Completing_the_circuit"><a class="zola-anchor" href="#Completing_the_circuit" aria-label="Anchor link for: Completing_the_circuit">Completing the circuit</a></h2> <p>The interesting part is what happened when the pieces started reinforcing each other.</p> <p>I feel more productive now than I have in a decade, but not in the generic "AI makes coding faster" sense. The multiplier hit hard because I already had the instinct. Twenty years of leaving breadcrumbs taught me that externalizing <em>is</em> how I think. What changed is that the notes are no longer write-only. What I learn in one session feeds the next. Review findings become development checklists. Development patterns become review criteria. The system gets better because I use it, and I get better because it remembers what I forget.</p> <p>The joke version is that I'm a hallucinating language model with a small context window. The real version is that I spent a career compensating for the shape of my mind, and only recently got tools that could close the loop.</p> <p>The workflow works because it was built around what I actually am, not around an imaginary, more disciplined version of me.</p> <hr /> <p><em>The tools referenced here are open source at <a class="external-link" rel="nofollow noreferrer external" href="https://github.com/butterflyskies">github.com/butterflyskies</a>.</em></p>