intertwingly

Deccan Queen on Rails is October 8–11 in Pune, and I've been asked what I'm bringing. The RubyConf talk demonstrated something: a Rails application with no server, then the same application as Rust, as Go, as a native binary out of Matz's compiler. Pune is where I want to say what I think that demonstration means, and the sentence I keep arriving at is one I would have laughed at a year ago.

Rails has been a fourth-generation language all along. We never noticed, because it was hiding inside a third.

What a 4GL was

In 1982 James Martin published Application Development Without Programmers, and for the next fifteen years an industry tried to make the title true. The tools were called fourth-generation languages. The generations were machine code, then assembler, then everything from FORTRAN to C to Ruby — the 3GLs, general-purpose languages in which you say how. A 4GL was the next rung: domain-specific and declarative, a language in which you say what. dBase and its descendants Clipper and FoxPro. Informix-4GL. Progress. Natural. FOCUS. PowerBuilder. Oracle Forms. And SQL, the one that survived, which is the tell.

They were astonishingly productive inside their domain, and the domain was almost always the same one: a database, some forms, some reports, some rules. A screen that took a week in C took an afternoon in a 4GL, because the 4GL already knew what a screen bound to a table was.

Most of them were interpreters. A few ventured into compilers — Clipper compiled the dBase language to native code and built a business on it; Informix shipped its 4GL both ways, compiled through C and interpreted as p-code — and the compilers were the expensive part. Every target was a team. A vendor whose revenue was runtime licenses had no reason to compile you onto anyone else's platform, and when the platform moved anyway — green screens to Windows, Windows to the web — a 4GL with one target and one vendor mostly didn't move with it. That is how they died. Not because declarative was wrong, but because the compiler was too expensive to keep alive while the world changed underneath it.

What Rails was

Twenty years of calling Rails a framework. A framework is a 3GL artifact: a library, in Ruby, that you call. That is true, and it is the less interesting truth.

Look at the eight lines of Article that opened Act III of the talk. has_many :comments. validates :title, presence: true. Those are not calls. They say nothing about query plans, or the runtime, or how an error gets rendered. They are statements about what an article is, and everything downstream — the schema, the form, the controller's shape, the route, the test — is derivable from them by convention. That is a 4GL statement. It is the same kind of line a Progress or Informix programmer wrote in 1990, and it took an afternoon for the same reason.

The difference, and it is the whole difference, is that Rails put its 4GL inside Ruby. You never had to learn a new language, and you never had to leave to escape it: the moment the conventions run out you write a def, and you are back in a 3GL, in the same file, three lines down. The boundary between the generations runs through the middle of every model. Nobody had to be told this, so nobody noticed. The escape hatch was so smooth that we filed the whole thing under "Ruby DSL" and moved on.

Roundhouse is what happens when you take the 4GL half seriously. The analyzer reads the declarations as a specification and compiles the specification; the 3GL half comes along as Ruby that has to be typed. And the ledger of what it cannot compile is, almost line for line, a map of where the 3GL took over: method_missing, define_method, the places where the answer exists only at runtime. The first Futamura projection, from the talk, is the same observation with a proof attached. Rails is the interpreter for the spec; specialize the interpreter to one program, and what remains is a compiled program.

Rails' homepage now says this from the other direction: "convention over configuration" pitched to coding agents, "token-efficient code that's easy for agents to write." A 4GL was always the answer to how little must the human say? Agents are asking that question again.

What changed

If Rails is a 4GL that is still an interpreter — and the 87% of a request the framework spends re-answering questions fixed at boot says it is — then the question of the 1990s is back. Can you afford the compiler?

The 4GL generation's answer was no, and the reason was economics, not theory. A target cost a team. Verifying the target cost another. And the incentive ran the wrong way.

Here is what the same job cost this year. Ten targets, one person, no team. Fifteen issues filed against Matz's compiler in a single week, all fifteen closed, four of them inside an hour. A fix to someone else's compiler in seventy-two minutes. Every commit to the RubyConf deck co-authored by an agent, and every fix of Matz's co-authored by his. This is not a story about a clever compiler. It is a story about what a compiler costs to write now, which is not much.

What it does not make cheap is knowing whether the compiler is right. That is where the cost moved, and the last month of posts is about little else: a DOM-diff of every route against real Rails, the application's own test suite run against the emitted code, a socket walk, a byte-level comparator, a person clicking around — a ladder in which every witness caught what the witness below it had certified. An agent will write you a code generator in an afternoon. The oracle that holds it to the framework it is imitating is the asset. The 4GL vendors of 1990 had neither the cheap generator nor the oracle. We have one, and we are building the other in public.

The bigger claim

There is a larger claim behind this one, and I'm not sure I'm ready to make it in Pune. A programming language was a contract between a human and a machine, and every generation of languages was a negotiation over how much the human had to say. The moment an agent writes most of the 3GL and a compiler consumes it, the thing the human actually reads, reviews, and argues about is the spec in the middle — the 4GL that was hiding in Rails the whole time. That is a change in what a language is, not merely which one we use.

Not with words

I would rather make this case with running code than with a post, and between now and October I'll be busy.

Roundhouse is open source: dual-licensed MIT / Apache-2.0. Issues and discussion welcome.