intertwingly

Yesterday, in Partly in the Right, I asked where the information comes from when an agent produces a result, and what checks it. Today I have a sharper version of that question, and a new piece of evidence to ask it about.

The as-if rule

Aaron Patterson's closing keynote at Rails World was about optimization, and he framed it with a rule language implementers know well. The C++ standard puts it in a footnote: an implementation "is free to disregard any requirement of this International Standard as long as the result is as if the requirement had been obeyed, as far as can be determined from the observable behavior of the program." C has the same idea without the name. The standard describes an abstract machine, lists the few things a conforming implementation must preserve, and leaves the rest to the compiler.

Aaron noticed the awkward phrase right away: "as specified by the standard, which we don't pretty much have in Ruby." Then he looked at optimizations through the lens of observable behavior. Every optimization, he said, has two sides: the thing that gets deleted, and whoever observes it. Some deletions are gray areas, like stack frames that vanish or allocations the JIT removes. You can observe them, and he asked the audience: "Do we violate the as-if rule? Do you care?" His answer was that he doesn't.

He ended on a different note. An hour in, he took on the argument that AI is just our next compiler, and you don't read the machine code a compiler produces:

But that's because the compiler made a promise to you. [...] The as-if rule, whatever it does, the behavior that you can observe matches the code that you wrote. [...] Now, your AI has not made this promise to you. There is no as-if rule for your English.

So he's going to keep reading his code. I think he's right about English. I also think there's a third option he didn't discuss, and two projects have now taken it.

Two ports, one oracle

Neither DHH nor I rely on a promise. We rely on an oracle. In both projects, "observable behavior" is defined by what the Rails app actually does, and the translation is checked against it mechanically.

This week 37signals made public Campfire in Rust. The first commit is dated September 26. The Rails app sits in a submodule, and a Playwright harness runs both apps side by side on seed data the Rails app generated. It compares the server HTML, the live DOM, the accessibility tree, every subresource, the Action Cable frames and the screenshots, pixel for pixel. When it was finished, the port passed all 5,018 cells of the full matrix. That's the as-if rule with the standard replaced by a running program: as far as can be determined from the observable behavior, and here the observable behavior was checked exhaustively.

Roundhouse compiles the same Campfire, pinned to the same upstream commit. CI boots both and compares the DOM, and Campfire's own test suite runs against the compiled binary.

Then the Rust port did what a C++ compiler may not do. Once parity was proven, its README says, "it now diverges from Rails where that makes it faster or better," and each divergence is listed under Known differences. There are about two dozen of them. Aaron's two sides of an optimization turn out to be exactly the right way to sort them.

The first row is legal under the as-if rule. Any compiler could do it, and a compiler is the right place for it. The last row isn't an optimization at all. Those are decisions about how the application should behave, and a translator has no business making them on the source's behalf.

Should they go back?

Suppose you were in a position to contribute to both once-campfire and Rails. Should the port's changes go back upstream?

For the behavior changes, I think the answer is clearly yes. They're improvements to Campfire, and the Rails version's users deserve them as much as the Rust version's do. Some are nearly free. The XSS is a bug in rails_autolink that affects every app using it. The search error, the directs#show 500 and the missing Edge icon are one-line fixes. One is already done: Rails main switched to Sec-Fetch-Site on December 12, 2025, eleven days after the Rails commit Campfire is pinned to. Backporting that one mostly means updating Rails and no longer rendering tokens that nothing checks.

The invisible row doesn't belong in Campfire, because none of it is Campfire's concern. It still has places to go. One item is already in Rails: the SQLite adapter has cached prepared statements for years, so the port rebuilt something Rails already had. Two would make good contributions to Rails. Running WAL checkpoints on a background thread fixes a stall every SQLite-backed Rails app has, where one unlucky request pays for the checkpoint. Action Cable encodes each broadcast separately for every subscriber, even when a Turbo stream's subscribers all receive identical bytes. The last one, splicing precompressed fragments into a gzip stream, belongs in whatever serves the page. That's a compiler's runtime, or a hand port, where the as-if rule allows it and the change is cheap. In Rails it would have to reach across Action View, Rack and the front server. The gray area is a judgment call, and it's the owners' call.

There's a second reason to backport, and it matters more than the first. Every behavior change left out of the source is a hole in the oracle. The Known differences list is a hand-written specification for everything the Rails app no longer defines. In yesterday's terms, it's information that isn't in the existing system, so someone has to supply it. As long as it lives only in a README, it is English, and there is no as-if rule for it.

Three ways to ship the next release

Assume the behavior changes are backported, so the Rails app once again defines everything a user can observe. Campfire keeps changing, and so does Rails. There are three ways to keep a fast version current.

Compile it. This is what Roundhouse does. Moving to a new Campfire commit means changing the pin and recompiling. Behavior changes in Campfire arrive for free, because they're in the source. A change in Rails, like the new forgery protection, gets implemented once in the compiler's runtime, and every compiled app picks it up. The same goes for an invisible optimization like gzip splicing. The costs are real. A compiler is a large thing to build. It covers a subset of Rails, and the subset is where the gaps are recorded. It depends on a toolchain, Matz's Spinel, that is still moving. And compiled output is slower than a hand port. How much slower is the subject of the next section. What it offers in return is the closest thing to Aaron's promise that this approach has. Roundhouse defines its subset as the apps that compile without error diagnostics. That's a statement about every app in the subset, tested across several apps, not a claim about one output.

Port it again. Do what DHH just did, on every release. The harness is the durable asset, and the Rust is disposable, which is Chad Fowler's regenerative idea taken literally. Every release gets a fresh port, checked against the same oracle. This works better with backporting than without it, because less has to be restated each time. But not everything can be backported. The invisible optimizations aren't in Campfire and shouldn't be, so each regeneration has to be told about them again, in English. And each regeneration produces a different ninety thousand lines, so there's nothing incremental to review. If you want to read your code, as Aaron does, you'd have to read all of it every time. It also takes days of agent time per release, which is fine for a major version and awkward for a security patch.

Maintain the Rust. Treat the port as the product and carry each upstream change over by hand, or by agent. It's the fastest at runtime, and the only option where each change arrives as a diff a person can read, which is Aaron's preference. But every Campfire change lands twice. The framework that Rails used to carry, around ten thousand lines of it for requests, sessions, cookies and the front server, plus a WebSocket implementation and an ACME client, is now 37signals' to maintain forever. And there's a slower cost that I think is the real one. Once the Rust version is what ships, nothing requires the Rails version to keep running. Backporting becomes optional, and optional work gets skipped. When the Rails app stops being run, the oracle stops being maintained. At that point the Rust is the source, the port is simply a program, and you're back in Aaron's world, reading the code.

That's also the one real advantage of this path: nothing has to go back upstream. No pull request to Rails, no waiting on anyone else's review or release. But the flip side matters more. An improvement to the maintained port helps one app. An improvement to Rails, or to a compiler's runtime, helps every app built on it. And the benefit runs both ways. When others contribute to the framework or the compiler, you get their work with no effort of your own. A maintained port gets none of that. Every checkpoint thread, broadcast cache or security fix that anyone else makes has to be noticed, and then ported by hand.

How fast is fast enough?

Both projects publish benchmarks against the same Rails app, at the same Campfire commit, deployed the way its own Dockerfile deploys it: Thruster in front of Puma, Puma's worker count from Campfire's own formula, Redis for the fragment cache. Here's what each measured, with the page served 16 requests at a time.

The first table is Roundhouse's nightly Campfire run. The second is the port's own final benchmark, taken at the commit that passed its parity gate. Read side by side, the port is about 1.4× ahead. That number isn't right, and there are three reasons why.

Different machines. The port was measured on four pinned hardware threads of a Ryzen 9 9955HX, a current part. Roundhouse's box is a Ryzen 5 3600 from 2019, with boost turned off so the numbers hold still, and the deployed lanes use all twelve of its threads. Dividing each side by its own Rails helps, but it assumes Rails and the fast version gain equally from a faster CPU, and they don't have to.

Compression. The port's harness asks for gzip, as a browser would. Roundhouse's harness uses wrk, which doesn't, so nothing on the Roundhouse side is compressing. That matters a great deal: in the port's report, gzip is 60–76% of the Rust app's CPU on the big pages. Its own "no gzip" column serves the room page at 8,973 requests a second on four threads. Per thread, that's more than four times what the compiled binary does on one. The two machines put the real gap somewhere between 2× and 4×, and I can't narrow it further from published numbers.

Optimizations that haven't gone back. I compared against the faithful build on purpose. The port's README headlines bigger numbers, 27× Rails on the room page and 44× on the messages page, but those come from later work, and much of it depends on the behavior changes above. Caching whole pages, compressed ahead of time, only works once pages stop carrying a fresh CSRF token. The faithful build does include the invisible optimizations that came first: fragment lookup before rendering, cached prepared statements, broadcasts encoded once, WAL checkpoints on their own thread. That's fair, because the as-if rule allows all of them. It also doesn't explain the gap, because Roundhouse's runtime already does the first three. Only the checkpoint thread is missing, and it affects writes, not the page loads measured here. Whatever the remaining factor is, it's more likely in the generated code and its runtime than in a missing trick. If Campfire adopted the header-based forgery protection that Rails main already has, whole-page caching would become an as-if optimization too, open to the compiler and the port alike.

The way to settle it is to run the port on Roundhouse's box, under the same harness, against the same database. The port says it's a drop-in replacement for the Rails app's database and cookies, which should make that easy. Until then, my honest estimate is this: compiled Rails serves pages 7–8× faster than Rails as deployed, and a hand port serves them perhaps 2–4× faster than that.

What Backus would say

John Backus had a number for this. When his team started on FORTRAN in 1954, the automatic programming systems of the day "slowed the machine down by a factor of five or ten," and programmers didn't trust them. In his history of FORTRAN he wrote that if FORTRAN produced "an object program only half as fast as its hand coded counterpart, then acceptance of our system would be in serious danger." That's why his team treated "the design of the translator as the real challenge, not the simple task of designing the language."

By that standard, Roundhouse isn't there yet, or is right at the edge. But the history is encouraging. FORTRAN shipped in April 1957. A survey of twenty-six 704 installations a year later found "over half of them use FORTRAN for more than half of their problems," and by that fall "more than half the machine instructions for these machines are being produced by FORTRAN." Backus won because the compiler got close enough, and the economics decided the rest: in 1954, programming and debugging accounted for "as much as three quarters of the cost of operating a computer."

Hand coding didn't disappear. It moved to the places where measurements said it was worth the effort. That's still true: 37signals this week merged a hand-written assembler engine for their terminal screensaver, 9.8× faster than the Rust it replaced. I expect Rails to settle the same way. Not compiler or port, but a compiled application with hand-tuned parts where measurement justifies them, and a runtime that takes every optimization the as-if rule allows.

The economics are different this time, though. Agents have made writing code cheap, so the price of a hand port is no longer the writing, which took 37signals days. It's everything after: following upstream, sharing improvements, and keeping the oracle alive. Those are the costs the three paths above divide differently.

A promise you can check

Aaron is right that your English makes you no promise. But the choice isn't between trusting an agent and reading every line. Two projects have now taken a third path: write down the observable behavior as a running program, and check against it mechanically. In C, the standard defines what's observable. Here the oracle does, and it stays authoritative only as long as it's kept complete. Every improvement that goes back upstream keeps it complete. Every one that doesn't leaves a hole, and someone has to fill it in English.

That's why I'd backport, whichever of the three paths you take. For the compiler it's required. For regeneration it's cheaper. For the maintained port it's optional, and that's the trap: skipping it keeps your improvements to yourself and keeps everyone else's improvements away from you.

Roundhouse is open source: dual-licensed MIT / Apache-2.0. Issues and discussion welcome. Campfire in Rust is at basecamp/once-campfire-rust.