intertwingly

The plan in Keep the Gems ended with "Falcon and Ractors last, once each request is cheap enough that the server model is what matters." Last week that point arrived. On the Campfire benchmark, the Ruby that Roundhouse emits from Campfire's Rails source was already ahead of Tobi's port on the room, messages, sidebar, search and post routes. It was behind on the three routes that do almost nothing: a user's avatar, a stylesheet, and the /up health check. Those measure the server and the layers around the app, and Tobi's port runs on Falcon.

As of the October 16 run, the benchmark page has a column for the same emitted application under Falcon:

Requests per second, eight workers, on the benchmark box's six cores. Under Falcon, the emitted app is well ahead of the port on the four routes that render pages and ahead on the stylesheet (which has gone both ways between runs). It's within 1% on the avatar and on posting a message, where the Puma lane is ahead. It's well behind on /up.

Three layers

The first gap was caching. Campfire's own deployment puts Thruster in front of Puma, and Thruster keeps any response marked Cache-Control: public in memory. Tobi's port does the same inside the process. Roundhouse's output had no cache at all, and its stylesheets carried no Cache-Control either, because it never read config.public_file_server.headers. Both are fixed: the header is read from the app's config, and the CRuby output now has Thruster's cache rules in process. Under Puma that took avatar from 14,610 to 45,333 requests a second.

That's where Puma stopped. A hello-world app under Puma on the same box serves about 54,000 requests a second; under Falcon, 106,000. Cached responses were already near Puma's ceiling.

Moving to Falcon added less than expected: avatar went from 45k to 46k. Profiling a single cached request in process showed why. The cache lookup took 3 µs. Going through Falcon's Rack adapter took 17 µs, about 14 µs of it building the Rack environment and converting the response headers back. Tobi's port doesn't use Rack at all; it gives Falcon a Protocol::HTTP application directly.

So the emitted tree now includes a falcon.rb. It answers cache hits before the Rack adapter runs, and passes everything else through to the unchanged config.ru. That's the jump to 75,552 on avatar.

The GVL

Falcon's first numbers for posting a message had the right median and a terrible tail: 99th percentile 360–716 ms, against about 50 ms for Puma. Under Falcon, each worker process serves all its requests from one event loop on one thread. Any other Ruby thread in that process that holds the global VM lock stalls every request at once, for up to the 100 ms time slice. Setting RUBY_THREAD_TIMESLICE=10 brought the tail down to 55–85 ms, which confirmed the cause.

There were two such threads. One ran background jobs: posting a message queues web push delivery and broadcasts. Under Falcon that loop now runs as a task on the worker's own event loop, where a job delays other requests by its own fraction of a millisecond instead of a time slice. The other was the SQLite WAL checkpointer, which holds the GVL for the whole checkpoint. It now runs in a process of its own that serves no requests. The tail is now 77 ms, against 64 ms for Puma and 63 ms for Tobi's port, so there's still a little to find.

Puma has the same threads, but there the other request threads keep running. It's a useful lesson for anyone moving a threaded Rails setup to a fiber server: look at which background threads come with you.

What's missing

• /up is still at 37,586 against Tobi's 61,378. His port checks for /up before routing and returns a constant. The emitted app goes through its router (now indexed at boot rather than scanned), takes a database connection, and adds a request ID, as Rails does. That overhead is general dispatch cost, and reducing it will speed up every route, not just this one.
• WebSockets. Action Cable in the emitted app hijacks the socket the way Puma expects. Under Falcon, /cable doesn't work yet, so the WebSocket section of the benchmark still runs Puma.
• It's not a default. The emitted Gemfile names Puma; the benchmark adds Falcon itself, at the version Tobi's port uses.
• Ractors aren't implemented yet; these numbers come from forked Falcon workers, one process per core. Tobi's port gets the same parallelism from Ractors in a single process, which matters for memory and for sharing state like Action Cable subscriptions, not for raw throughput on this box. Memory shows it: the Falcon lane's processes take 1.2 GB together, Tobi's 0.7 GB. The sqlite3 and nokogiri pull requests for Ractor support are still open.

One data point for the Falcon and protocol-rack maintainers, offered as a measurement rather than a complaint: in a profile of cached requests through the Rack adapter, header handling was the largest cost. Protocol::HTTP::Headers#delete (by way of extract and partition) and String#downcase together took over half the samples. Apps that serve many small cached responses through Rack would benefit from making that path cheaper.

The code is in the roundhouse repository: falcon.rb and runtime/thruster_cache.rb in the CRuby overlay, and the ruby-falcon lane in scripts/bench-campfire.