intertwingly

Yesterday's post ended with Roundhouse's futamura target at 1.00× Rails on the Lobsters benchmark, and a ceiling for it of 2.5–3×. A day of measuring moved both. This post explains the name first, since the idea behind it is the whole plan.

Futamura projections

In 1971 Yoshihiko Futamura made an observation about partial evaluation. A partial evaluator, or specializer, takes a program and the part of its input that is known ahead of time, does every computation that depends only on that part, and writes out a smaller program, the residual program, that waits for the rest. Give a power function the exponent 3 and you get back x * x * x, with the loop gone.

Futamura noticed what happens when the program being specialized is an interpreter:

1. Specialize an interpreter with respect to the program it is about to run, and the residual program is that program, compiled. The interpreter's work of deciding what each piece of source means has been done once, ahead of time.
2. Specialize the specializer with respect to the interpreter, and you get a compiler.
3. Specialize the specializer with respect to itself, and you get a compiler generator.

These are the three Futamura projections. The image at the top of this post draws them as tombstone diagrams, the T-shapes compiler writers use to show a program standing on what runs it.

Rails is, operationally, an interpreter for your application. Your routes, associations, callbacks and templates are data it consults on every request: which filters apply to this action, what SQL this scope chain means, which partial this name refers to. None of those answers changes between requests. Specialize Rails with respect to your application and, by the first projection, what comes out is your application with the interpretation done ahead of time.

Roundhouse doesn't have an automatic specializer. It builds the first projection by hand, one specialization at a time, from an analysis that reads the whole application before it runs. What it can't decide stays residue, and the futamura target's rule is that residue runs on Rails, with your gems. The name records where the target is headed, not a claim that it's there.

Where specializing inside Rails runs out

The specializations so far work inside Rails: a static query chain handed to Rails' own statement cache, a preload tree planned once. They keep Rails' Active Record objects, so gems hooked into those objects keep working. Profiling Lobsters says that strategy has a ceiling near 2×, not the 2.5–3× I estimated yesterday. Even faster attribute reads inside Rails' own model classes would be worth 3–4% at most. Most of what is left of Rails' cost is the objects themselves, and they come off only when a whole request never builds them.

Roundhouse already writes a version of the application that never builds them: the ruby target, plain Ruby on Roundhouse's own small runtime, which runs this benchmark about 4× faster than Rails. The question was how to use it for some requests and leave the rest to Rails.

Ruby::Box

Both have to run in one process, and they can't share a namespace. Roundhouse's runtime defines its own ActiveRecord, ActionController and ActiveSupport, which collide with Rails' own.

Ruby::Box is what makes this possible. It arrived in Ruby 4.0 as an experimental feature, enabled by starting Ruby with RUBY_BOX=1. A box is a separate space for classes, modules, globals and loaded files. Code loaded into a box defines its constants there; the core classes are shared, but a monkey patch made in one box isn't seen from another. Roundhouse's output goes into a box, and its ActiveRecord is not Rails'.

Whole routes

The futamura target now ships that output alongside your application. A middleware just after Rails' session store hands a GET request to it when the analysis can vouch for the whole route, and Rails serves everything else. In Futamura's terms, the residue is now chosen per route: a route is either fully specialized or fully Rails.

Vouching is a reachability question. Start from the action, follow the calls through its filters, views, models and helpers, and if any of them reaches code that Roundhouse reports as an error, meaning code it might translate wrongly, the route stays on Rails. For each route it leaves to Rails, the build says why, with the call path to the first error:

GET /top home#top: reaches 1 error(s), first app/controllers/home_controller.rb:267
    `>` with incompatible operand types: Integer | String | ... | nil? > Integer
    via HomeController#process_action -> HomeController#top

That turns Roundhouse's list of errors for an application into a list of routes. Fixing an error moves the routes it was holding back.

Three things cross between the two applications, and no model object ever does. Rails' session goes in as a hash and comes back with whatever changed, so logins and CSRF tokens work across the boundary. The flash goes the same way. And the database connection is Rails' own, lent for the request. It matters that it's the connection and not the file: a box loads its own copy of every C extension, so if the boxed code opened the database itself there would be two copies of SQLite on one file in one process, which SQLite warns against. Through Rails' connection there is one, and the boxed code also sees Rails' transaction, so in a test it reads the test's fixtures.

Measured

On Lobsters, the analysis clears 48 of 84 GET actions, 40% of the benchmark's requests:

That is Ruby master with YJIT, on a different machine from yesterday's tables. The routes served from the box run 2.3–4.7× faster than Rails; the story page goes from 14.1 ms to 3.0, the comments page from 17.7 to 4.6. The pass as a whole gains only 1.17×, because starting Ruby with RUBY_BOX=1 slows Rails itself by about 19% before any box is created, and every route left to Rails pays that. When an earlier prototype sent every GET to the box regardless of the analysis, the pass took 377 ms, 3.3× Rails. The difference between 1.17× and 3.3× is the error list.

Real-blog's own test suite passes on the output with its GET requests served this way, and all 26 Lobsters pages we compare render the same content as Rails.

Ruby 4.1

This needs Ruby 4.1. On 4.0.x, Rails crashes under RUBY_BOX=1 as soon as it loads net/imap; Ruby master doesn't. That makes Ruby::Box a hard dependency of this part of the target, and its cost to Rails the next thing worth understanding. The other open items are writes, which stay Rails' so that gems' model callbacks keep firing, and a cache the two sides don't yet share.

How it works, with the full numbers, is in the specialization notes.