intertwingly

I built these for Roundhouse, which compiles Rails applications, and Campfire needs all three: Action Text sanitizes every message through rails-html-sanitizer, which is built on Loofah, which is built on Nokogiri and Crass. But none of them need Rails, and I suspect they are more useful on their own than inside a compiled Rails app.

What they are

• spinel-nokogiri parses HTML5 (with gumbo), HTML4 and XML over the same libxml2, with the same patches, that Nokogiri ships. It supports CSS and XPath queries, editing and serialization.
• spinel-loofah is Loofah's HTML sanitizer: its safe lists, its built-in scrubbers (:strip, :prune, :escape, :whitewash, :nofollow, …), custom scrubbers, and scrubbing of CSS style attributes.
• spinel-crass is Crass, the CSS tokenizer and parser Loofah uses.

Spinel is Matz's ahead-of-time compiler for a subset of Ruby, and spin is its package tool. Here's a complete program:

require "loofah"

# Untrusted HTML on stdin, safe HTML on stdout.
html = $stdin.read
puts Loofah.html5_fragment(html).scrub!(:strip).scrub!(:nofollow).to_s

spin new sanitize && cd sanitize
spin add loofah --git https://github.com/rubys/spinel-loofah --ref v0.1.0
# put the program above in bin/sanitize.rb
spin build
echo '<p onclick="x()">Hi<script>alert(1)</script> <a href="https://example.com">l</a></p>' \
  | ./build/bin/sanitize
# <p>Hialert(1) <a href="https://example.com" rel="nofollow">l</a></p>

That's a standalone executable. The libxml2 and gumbo sources are compiled into it, so the machine that runs it needs no Ruby, no gems and no libxml2:

Each time is the average of 200 runs on the same small document, so it's mostly startup. Both programs print identical output. That makes these a good fit for command-line filters, git hooks, static-site build steps, feed readers, and anything else that runs once per input.

How I know the answers are right

These aren't approximations of the gems. Loofah and Crass are the gems' own source. Where Spinel can't compile something, there's a rewrite: nine in Loofah and two in Crass. Each one is marked in the code, explained in the README, and visible as a diff against a first commit that is the gem, verbatim.

The tests are the gems' own inputs. I ran Loofah's test suite under CRuby with a recorder on Loofah's entry points, and kept every String the suite passed in: about a thousand HTML fragments, 17 documents, 91 style values and 212 URIs. Each fragment goes through both parsers and every built-in scrubber, in the compiled package and in the real gem. That's about 25,000 lines of output, and they match byte for byte. Crass is held to Simon Sapin's css-parsing-tests, and three real stylesheets must parse and serialize back to the same bytes. Nothing in the expected output was written by hand.

The corpus found real differences, and all of them are fixed:

• In spinel-nokogiri: markup was passed to the parsers with strlen, so an embedded NUL ended the document. content= on a text node encoded characters the gem stores raw. node["href"] found an SVG xlink:href, so Loofah's :targetblank added target="_blank" to SVG links. And a copied document came back as a plain Nokogiri document.
• In Spinel itself, seven defects. Two were silent miscompiles: a constant defined inside class << self read as nil, so Loofah deleted <, > and & from script bodies instead of escaping them; and super from a class method built the parent class. I filed six with small repros, and each was fixed upstream the same day. The seventh I fixed myself, in a pull request Matz merged, along with a second one that lets require "cgi/escape" work.

What's different

• Input is a String. The gems also accept an IO, and document parsing here takes the markup alone, without parse options.
• Large numbers in CSS. A number too big for 64 bits (width: 12345678901234567890px) scrubs to exactly the gem's output. Crass reads its value as a Float, where CRuby would use a Bignum.
• The language is a subset. Spinel compiles a subset of Ruby, so your own code has to stay inside it too.

The READMEs list the details.

Try them

They're version 0.x, with index entries pending in spin-index. Until those merge, add them by git URL as shown above. If you find an input where the compiled package and the gem disagree, that's a bug; please file it with the input.

Next is rails-html-sanitizer, compiled unmodified on top of these. Then Campfire's rich-text messages go through the same code in both lanes, the Ruby one and the compiled one.