Field notes — the blog

Notes from inside the system.

Specific write-ups on decisions that moved the needle: architecture trade-offs, delivery pipelines, and leading engineers who are better than you at their part of it.

              {{ totalPosts }} notes
              {{ totalMinutes }} min total
              Monthly
            

/

                Latest
                {{ featured.date }}
                {{ featured.cat }}
                {{ featured.read }}
              

{{ featured.title }}

{{ featured.excerpt }}

Read the note →

// before → after
{{ row.text }}

{{ listHeading }}

{{ resultCount }}

                {{ post.cat }}
                {{ post.date }}
              

{{ post.title }}

{{ post.excerpt }}

{{ post.read }}

Nothing matches that.

Try a different term, or clear the filters.

The architect's dispatch

One note a month. Nothing else.

A single deep write-up on a system I worked on: the trade-off, the numbers, what I'd do differently.

            {{ article.cat }}
            {{ article.date }}
            {{ article.read }}
          

{{ article.title }}

{{ article.excerpt }}

{{ part.text }}{{ part.text }}{{ part.text }}

{{ b.alt }} — {{ b.caption }}

{{ b.label }}

{{ h.text }}

{{ cell.text }}

{{ b.label }}

{{ link.label }} {{ link.note }} {{ link.host }}

{{ b.caption }}

{{ b.text }}

{{ b.text }}

{{ b.label }}

{{ line.text }}

{{ item.no }}

{{ part.text }}

{{ part.text }}

{{ part.text }}

Takeaways

▖ {{ t }}

Tags

{{ tag }}

← Newer

{{ prevPost.title }}

Older →

{{ nextPost.title }}

Related notes

{{ post.cat }}

{{ post.title }}

{{ post.read }}

Vibe coding works. Here's the side-effects leaflet nobody read.

AI · Sep 2026 · 17 min read

Eighteen months in, the evidence is finally in and it is not the argument either side wanted. The throughput is real. So is a 441% jump in time spent in review, a 17-point drop in what learners retain, a bug bounty shut down by slop, and a framework company that lost 80% of its revenue while its downloads hit a record. Here is the whole leaflet, with the dosage I actually use.

Every medicine that works has a leaflet, and the leaflet is long precisely because the medicine works. Nobody writes four pages of side effects for a sugar pill. Vibe coding is at that stage now. It stopped being a question of whether it does anything some time last year. The question worth asking in September 2026 is what else it does, to whom, and at what dose.

The timeline is short enough to hold in your head. Andrej Karpathy named it in February 2025: describe what you want, accept what comes back, do not read the diff. By November it was Collins' word of the year. By early 2026 Karpathy himself was calling it passé and offering a new name, agentic engineering, for the version where you orchestrate agents with specs, tests and review instead of vibes. When the person who coined a term retires it within about a year, it is worth asking what he saw.

What he saw, I think, is the data. There is a lot of it now, from randomised trials, from telemetry across tens of thousands of developers, and from the open-source projects on the receiving end, and it finally lets us write the leaflet properly. So here it is, in the order a leaflet uses: what it is for, the common effects, the serious ones, the effects on people around you, the interactions, and when to stop.

A mock medicine leaflet titled Vibe Coding, patient information, read all of this leaflet before you ship. Indicated for prototypes, internal tools, market tests and code you will delete. Common side effects: feeling faster than you are, measured as 19 percent slower while believing 20 percent faster; the work moving into review, median time in review up 441 percent; code that grows and never gets tidier, with refactoring down 70 percent and duplication up 81 percent. Uncommon but serious: skills that never form, 17 percent lower quiz scores for learners; open databases and borrowed agents. Effects on others: maintainers drowning in generated reports, and documentation traffic collapsing. Interactions: usage-based bills. Stop taking if the codebase is mature, if you are learning something new, or if nobody will review the output. — Every figure on this leaflet is sourced below. None of them says stop. All of them say dose.

What it is indicated for

Start with the part the sceptics skip, because a leaflet that only lists side effects is not a leaflet, it is a warning label, and people rightly ignore those. The gains are real and they are now measured, not just claimed. A field experiment across 4,867 developers at three companies found about 26% more completed tasks a week with an assistant. Faros AI's telemetry across 22,000 developers, published in April 2026, found task throughput per developer up 33.7% and epics completed up 66% under high AI adoption.

And the use cases where the side effects barely matter are genuinely large: a prototype to put in front of five customers, an internal tool three people will use, a calculator, an intake form, a market test, a script you will run once. For those, vibe coding in its original, do-not-read-the-diff sense is not reckless. It is the correct amount of engineering. I wrote about the disposal ending for exactly this code: it is fine as long as the deletion actually happens.

Common side effect: feeling faster than you are

The most-cited finding in this whole field is still the METR trial from July 2025. Sixteen experienced open-source developers worked real tasks in repositories they knew well, with and without AI. Before starting they predicted AI would make them 24% faster. Afterwards they believed it had made them 20% faster. Measured, they were 19% slower.

Three horizontal bars around a zero line for the METR 2025 trial of experienced open-source developers. Expected before starting: 24 percent faster. Believed afterwards: 20 percent faster. Measured: 19 percent slower, drawn in red on the other side of the zero line. Below, a note on the February 2026 follow-up: METR could no longer measure cleanly because 30 to 50 percent of developers declined to submit tasks they did not want to do without AI. — The gap between the two sides of the zero line is the side effect. The follow-up is arguably the stronger finding.

The honest update matters here, because this number gets quoted as if it were permanent. METR ran a larger follow-up from August 2025 and, in February 2026, published the fact that it could no longer produce a clean answer. The point estimates had moved towards a speed-up. But between 30% and 50% of developers said they were not submitting some tasks because they did not want to do them without AI, which skews exactly the comparison the study exists to make. METR called its own new numbers very weak evidence.

The most interesting result of the follow-up is not a speed-up or a slowdown. It is that a third or more of the developers would no longer agree to work without it.

That is not a scandal, and I would not call it addiction. It is what adoption of any good tool looks like. But it does mean the feeling is now the main thing most teams measure, and the feeling has a documented bias. The Stack Overflow survey found 66% of developers citing answers that are almost right, but not quite, as their top frustration, and 45% saying debugging generated code takes longer than writing it. Both are costs you feel as effort rather than as time, which is precisely why they do not show up in the sense of speed.

Common side effect: the work moves into review

This is the one that I think will define 2026, and the Faros numbers are the clearest picture of it so far. The same telemetry that showed throughput rising showed what happened downstream. Median time a pull request spends in review: up 441.5%. Bugs per developer: up 54%. Incidents per merged pull request: up 242.7%. And pull requests merged with no review at all: up 31.3%.

Two groups of bars on the same linear scale, from Faros AI's 2026 telemetry across 22,000 developers. Upstream, what the dashboard shows: pull request merge rate per developer up 16 percent, task throughput up 34 percent, epics completed up 66 percent. Downstream, where it lands: pull requests merged without any review up 31 percent, bugs per developer up 54 percent, incidents per pull request up 243 percent, and median time in review up 441 percent, a bar that runs almost the full width of the chart. — Drawn to the same scale on purpose. The upstream bars are what gets presented in the quarterly review.

Faros sells engineering measurement, so treat the precise figures with the scepticism you would give any vendor report. But the direction matches everything else. Google's DORA report for 2025 surveyed nearly 5,000 technology professionals and landed on the word amplifier: AI makes a strong delivery system faster and a weak one faster at producing instability. And it matches the shape I described in the bottleneck moved to review and nobody staffed for it: generation got cheap, verification did not, and the queue formed in front of whoever is most senior.

The 31% rise in unreviewed merges is the number I would put in front of any engineering manager. It is what a review queue does when nobody budgets for it. The code does not wait politely. It finds a way around.

Common side effect: code that grows and never gets tidier

GitClear has now looked at 623 million code changes from 2023 to 2026, and the pattern is consistent enough to state flatly. Moved lines, their signal for refactoring, are down 70% on 2022. Duplicated code blocks are up 81%. Constructs that swallow errors rather than handle them are up 47%. Code rewritten within two weeks of being written is up 15%.

None of those are bugs. Every one of them is a codebase getting harder to change. An agent is superb at adding the next thing and has no reason to subtract anything, because nothing in a build-until-it-works loop rewards deletion. Consolidating three near-identical helpers into one is exactly the work that used to happen when a human had to read all three, and that reading is the step vibe coding removes by definition.

The agent adds. Nobody subtracts. Every metric that measures reuse is falling, and every one that measures repetition is rising.

The August 2026 state-of-the-art review of this literature offers the explanation I find most convincing for why the productivity studies disagree so wildly: the gains are real on new code and shrink, or reverse, on mature codebases. That fits the METR result, which was run on large, old repositories, as well as it fits every glowing demo, which is run on an empty folder.

Uncommon but serious: skills that never form

Early in 2026 Anthropic published a randomised trial that I think every engineering lead should read, partly because of who published it. Fifty-two developers, mostly junior, learned an async Python library none of them knew. Half had an AI assistant. Minutes later, on a quiz about concepts they had just used, the AI group averaged 50% and the hand-coding group 67%, a gap of nearly two letter grades. The AI group was not meaningfully faster. The biggest gap was on debugging questions.

The finding that makes this useful rather than depressing is the one underneath: how people used the assistant mattered more than whether they had one. Participants who asked it for explanations, asked conceptual questions, or wrote the code themselves and used AI to check their understanding held on to far more. Delegation was the problem. The tool was not.

Uncommon but serious: the open database and the borrowed agent

I have written two whole notes on this, so briefly. Veracode found 45% of generated code samples introduced an OWASP Top 10 flaw, and bigger models were not safer. Scans of deployed vibe-coded apps keep finding the same thing: a database anybody can read, because the architecture these tools generate has no middle tier to hold the access check. And in August 2025 an npm package showed that the agent itself is now attack surface: malware that, instead of searching your laptop, asked your AI CLI to.

Side effects on the people around you

This is the section that leaflets for medicines never need and this one does. Some of the costs of vibe coding land on people who never used it.

On 31 January 2026, curl ended the bug bounty it had run for six years. For most of that time more than 15% of submissions had turned out to be real vulnerabilities. In 2025 that fell below 5%, as generated reports — fluent, confident, plausible and wrong — arrived faster than a small security team could disprove them. Daniel Stenberg's stated aim was to remove the incentive to submit them. A reward system built for humans could not survive a population that could produce reports at no cost.

Three weeks earlier, Tailwind Labs laid off three of its four engineers. Revenue was down around 80%. Documentation traffic was down 40%. And the framework itself was more popular than ever, at roughly 75 million downloads a month, largely because AI tools generate Tailwind by default. The business ran on developers visiting the docs and discovering the paid products. Agents read the docs on their behalf, and nobody visits any more. Adam Wathan made the announcement while declining a pull request that would have made the docs easier for LLMs to read.

The code the agent writes for you is assembled from open source whose maintainers are paid by attention the agent no longer sends them.

I do not think this is anybody's fault in particular, and I do not think it resolves itself. It is the most important hot topic in this whole field that is not about code quality at all, and it is the one where individual behaviour genuinely helps: if a library shows up in every generated project you ship, it is now a dependency you rely on and do not pay for, and the attention that used to pay for it is gone.

Interactions

With your bill. Every major tool has moved to usage-based pricing of some kind, and vibe coding is exactly the usage pattern that makes it hard to predict: long agent runs, retries, large contexts, extended reasoning billed as output. Cursor's switch from request counts to a usage allowance in June 2025 was communicated badly enough that it offered refunds for unexpected charges between 16 June and 4 July. The lesson generalises: if you cannot see what a turn costs, you are not in control of what a month costs.

With trust. In Stack Overflow's 2025 survey, 84% of developers were using or planning to use AI tools, 3.1% highly trusted the output, and 45.7% distrusted it. And 72.2% said vibe coding was not part of their professional work. That is an unusual shape for a technology: use going up while trust goes down. It usually means people have found the thing useful and are quietly compensating for it, which is fine right up until the compensation is the part nobody budgeted.

The rename is the tell

The loudest argument online this month is some version of a line doing the rounds among developer writers: vibe coding isn't the problem, calling it engineering is. I agree with it, and I think Karpathy's rename concedes the same point from the other direction. Agentic engineering is vibe coding plus the things this leaflet lists as remedies: specs, tests, review, checkpoints, and a human who owns the result.

That is not a new discipline. It is the old one, with a much faster and much less reliable junior colleague on the team. Everything that used to make a team good at absorbing a fast junior — small changes, real review, tests that assert behaviour, someone who knows why the code is shaped the way it is — is what makes a team good at this. The DORA word was amplifier. It amplifies whatever you already had.

Dosage: what I actually do

Each side effect, how you notice it, and what I do about it
Side effect	How you notice	What I do
Feeling faster than you are	Nobody can say what shipped faster, only that it feels faster	Measure lead time and incidents per change, never the feeling or lines generated
Work moves into review	The review queue grows and unreviewed merges creep up	Cap the size of generated changes and budget review time as real work, owned by name
Code grows, never tidies	Three helpers that do the same thing; nothing ever gets deleted	A deliberate subtraction pass after every feature, and duplication tracked in CI
Skills that never form	A junior who shipped it cannot explain it or debug it	Learning work in explain mode: the human writes, the model questions and checks
Open data, borrowed agent	You have only ever tested with one account, and bypass mode is on	A second test user before any public URL; bypass disabled in managed settings
Effects on maintainers	One library appears in every project you ship	Pay for what your agent keeps choosing: sponsorship, the paid tier, or a real contribution
The bill	The monthly figure surprises you	A spend cap and a per-session cost readout, reviewed weekly like any other bill

The first row does the most work. Almost every argument about vibe coding is two people comparing feelings, and the only way out is to measure the thing you actually care about. For most teams that is how long a change takes to reach production and how often it causes an incident. If both are improving, you are taking the right dose. If throughput is up and incidents are up faster, the dashboard is showing you the upstream bars and hiding the downstream ones.

When to stop taking it

When the codebase is mature and the change touches something old. That is where the evidence says gains shrink or reverse, and where one plausible wrong edit is most expensive.
When the point of the task is to learn. Use the model as a tutor that asks you questions, not as a contractor that hands you answers, and accept that it will feel slower, because it is.
When nobody will review the output. Generated code without a reviewer is not a speed-up; it is a deferred incident with a timestamp you do not get to choose.
When it touches money, personal data or credentials. Not because the model is worse there than a tired human, but because the failure is silent and the blast radius is somebody else's.

None of that says stop. The leaflet for anything that works is long, and the people who read it are the ones who get the benefit without the side effects. The only way to get this badly wrong is to keep taking it at the dose that felt good in the demo, on a codebase that stopped being a demo a year ago.

References

METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity — 16 developers, real tasks in their own repositories: expected 24% faster, believed 20% faster, measured 19% slower.
METR, We are changing our developer productivity experiment design (Feb 2026) — 57 developers, 800+ tasks. 30–50% withheld tasks they did not want to do without AI; METR calls the new estimates very weak evidence.
Anthropic, How AI assistance impacts the formation of coding skills — 52 mostly junior developers learning Trio. AI group 50% vs hand-coding 67% on the quiz; largest gap on debugging; explanation-seeking users retained more.
Faros AI, AI Engineering Report 2026: The Acceleration Whiplash — 22,000 developers, 4,000+ teams. Throughput +33.7%, epics +66%; median time in review +441.5%, bugs per developer +54%, incidents per PR +242.7%, unreviewed merges +31.3%.
GitClear, The Maintainability Gap: 2026 AI code quality research — 623M code changes, 2023–2026. Refactoring moves −70%, duplicated blocks +81%, error-masking constructs +47%, two-week churn +15%.
Michels et al., Vibe coding: practice, performance, productivity, and risk (Aug 2026) — State-of-the-art review. Gains real on new code, shrinking or reversing on mature codebases; self-report diverges from independent measurement.
DORA, State of AI-assisted Software Development 2025 — Nearly 5,000 respondents. AI adoption at 90%; AI as an amplifier of existing strengths and weaknesses.
Stack Overflow Developer Survey 2025, AI — 84% using or planning to use AI; 3.1% highly trust it, 45.7% distrust; 66% frustrated by almost-right answers; 72.2% say vibe coding is not part of their work.
The Register, Curl shutters bug bounty program to stop AI slop — Confirmed-vulnerability share fell from above 15% to below 5% in 2025; the bounty ended on 31 January 2026.
DevClass, Tailwind Labs lays off 75 percent of its engineers — Revenue down around 80% and docs traffic down 40% while downloads hit record levels.
The New Stack, Vibe coding is passé — Karpathy proposes agentic engineering: orchestrating agents with oversight, rather than accepting code unread.
Cursor, Clarifying our pricing — The move to usage-based Pro pricing, and refunds for unexpected charges between 16 June and 4 July 2025.
Veracode, 2025 GenAI Code Security Report — 45% of generated code introduced an OWASP Top 10 flaw; larger models were not safer.

Takeaways

The gains are real and measured: about 26% more completed tasks in a large field experiment, and 33.7% higher task throughput in Faros's 2026 telemetry. The argument is about dose, not about whether it works.
METR measured experienced developers 19% slower while they believed they were 20% faster. Its 2026 follow-up could not measure cleanly because 30–50% of developers would not do some tasks without AI.
The work moves into review: median time in review up 441.5%, incidents per pull request up 242.7%, and unreviewed merges up 31.3% in the same data that showed throughput rising.
Codebases grow and stop getting tidier: refactoring down 70%, duplication up 81%, error-masking up 47%. The agent adds and nothing subtracts.
Gains appear real on new code and shrink or reverse on mature codebases, which reconciles most of the contradictory studies.
Learners using AI scored 17 points lower on concepts they had just used, worst on debugging. Asking for explanations instead of answers largely closed the gap.
The commons pays: curl ended its bug bounty when valid reports fell below 5%, and Tailwind lost 80% of its revenue while its downloads hit a record.
Usage-based billing and falling trust are the interactions: 84% use AI, 3.1% highly trust it.
Karpathy's rename to agentic engineering concedes the point: specs, tests, review and ownership are the remedies, and they are the old discipline applied to a faster, less reliable colleague.
Measure lead time and incidents per change, not the feeling. If throughput is rising and incidents are rising faster, you are looking at the upstream bars only.

All notes · Shehzad Aslam