intertwingly

It’s just data

Brief history of the ASF Board Agenda tool

The gold standard of server side web applications is Model, View, Controller.  Early versions of this tool was not written that way: it was a CGI script that grew like a weed.  Over time, some JQuery effects were added.

The first major rewrite was done using Angular.js and Bootstrap.  These frameworks enabled me to do things I had never done before.  It also required me to write code that watched for changes, and to ensure that changes were applied in place (specifically arrays and hashes could not be replaced, they had to be updated).

While Angular.js used terms like Directives, Filters, and Services, the overall effect was to impose a structure on the client side application.  As with most things, this structure was both constraining and freeing.

The current rewrite replaces Angular.js with React.js.  Gone is all watches and the need to update things in place.  In its place is a policy of “rerender everything” whenever an event (a keystroke, a mouse click, a server side event) occurs.  With React.JS, rerendering everything is efficient as React computes a delta and then only applies the delta to the DOM.  React.JS does provide a suggested architecture, namely Flux, that minimizes the need to rerender everything, but in practice I have not found that necessary.

To illustrate, if you bring up the “Call to order” page and press and hold down the right arrow key, every page of the agenda will be flashed up and promptly replaced.

The overall resulting flow is as follows: when a page is fetched the response starts out with a pre-rendered representation (simple HTML), followed by the scripts needed to produce that page, followed by the data used by those scripts.  This ensures that the data is presented promptly, then become reactive to input and events.

The resulting architecture isn’t MVC on either the client or the server.  Instead, V and C get mushed together, and a unified client/server event stream is added.

Events are received from the server using Server Sent Events.  This is widely implemented, and has a solid polyfill for browsers (most notably, IE) that haven’t implemented this standard.  Its one way data flow is a good fit for React.js.

Events are generally triggered by actions on a client browser window somewhere (typically a mouse click) resulting in a HTTP GET or POST request being sent to the server, but can also be triggered by file system changes on the server (example: a cron job does a svn update, which causes the agenda to contain new data).

A single event-stream is maintained per browser, and that process is responsible for propagating updates to all tabs and windows.  Events can be sent to all clients, or only clients authenticated with a given user id.  This enables my pending updates to be immediately reflected on all of my tabs and windows but not affect others.  The result of an event is to update one or more models, and then trigger a re-render.

Models on both the client and server are simple classes.  Class methods operate on the entity as a whole (example: write the whole agenda to disk on the server, or provide an index for the agenda on the client).  Instance methods refer to an individual item (example: an agenda item).

What’s left is React Components on the client and actions on the server.

React components have a render method.  That method has full (read-only) access to client models, and can do if statements, iterate over result, and (generally minor) computations.  More extensive computations should be refactored to other methods in the component when limited in scope to a single component, or to the client model otherwise.  The one limitation that is enforced is that render methods can not directly or indirectly change state.  A predefined life-cycle is defined.  Other methods can be added, for example methods to handle onClick events.

These methods can trigger HTTP POST and GET requests (the convenience method I provide for the latter is called fetch instead).  These run small scripts on the server that may update models, generate events, and return JSON.

Taken together, the current implementation is a lot more fun to develop and easier to maintain than prior versions.  As an example, if it were decided that the moment the secretary clicked the ‘timestamp` button on the 'Call to order’ page, all comment buttons are to be removed from all windows and all comment modal dialogs are to be closed, this could be implemented using a single if statement as the event is already propagated, and a re-render is already triggered.  All that would be required is to change the conditions under which the comment button appears.

The board agenda tool has been tested on Linux, Mac OS/X, Vagrant, and Docker.  It contains a suite of tests.


Spartan Test Results

I replaced IE results with Spartan results in my urltests.  Other than the user agent string, nothing changed.

Following are selected examples where three out of four of the top browsers agree, identified by the odd browser out:


Ruby2JS 2.0

I’ve released Ruby2JS version 2.0.  Key new features:

The Whimsy Agenda rewrite-in-progress (previously based on Angular.js, now being rebased on React.js) can be used to explore both of these features.


React.rb updates

I’ve made a number of updates to the demos.  The tutorial demo has been updated to do server side rendering.  This means that it is able to be used by clients which either don’t support or have turned off JavaScript. 

The second demo is a calendar.  Unlike the tutorial which is a single file, this application is organized in a manner more consistent with how I expect projects to be organized.

...


DSL for JavaScript

W

Jeremy Ashkenas: “work towards building a language that is to ES6 as CoffeeScript is to ES5”… close, but—do it for [ES6+HTML+CSS], and you’ll win ;)

It occurs to me that there is a shortcut available.  Let a library like React replace [ES6+HTML+CSS].  Then build a DSL for that library.

...


Web Components

Brian Leroux: ES6 and Web Components

My take is that this talk lumps React in with others based on when it was introduced; but that it is fundamentally different from, say Angular.js as Angular.js is from jQuery.

...


Email addresses

I have been telling all non-IBMers to not use my ibm.com email address for years, but this advice is routinely ignored.  I’ve repeated the reaons behind why I ask this enough times that it makes sense for me to post the reasons in one place so that I can point to it.

...


React.rb

Having determined that Angular.js is overkill for my blog rewrite, I started looking more closely at React.  It occurred to me that I could do better than JSX, so I wrote a Ruby2JS filter.  Compare for yourself.

...


RFC 3986bis

URL parsers consume URLs and generate URIs.  Such URIs are not RFC 3986 complaint.  I’d like to fix that.

...


URL Work Status

I have test results that show that there is much work to be done.

The most likely path forward at this point is to get representatives from browser vendors into a room and go through these results and make recommendations. This likely will happen in the spring, and in the SF Bay Area. With that in place, I can work with authors of libraries in popular programming languages to produce web-compatible versions. This work will take the form of bug reports, patches, or — when required — authoring new libraries.

...


Ununzippable Modern.IE

I’ve downloaded the multi-part zip archive for IE11 on Win10 for VirtualBox on OS/X from modern.ie.  I’ve downloaded the single-file archive on both OS/X and Linux.  I’ve verified the md5 signatures for each.  Yet each time, when I try to unzip the result, I fail.

...


New PhantomJS and Capybara fan

While I’m clearly late to the party, I’ve already become a huge fan of capybara and phantomjs.  I’m now using both with my previously mentioned blogging software rewrite.

My original intent was to aggressively prune unnecessary function with the intent of producing a more maintainable result, but with the ability to have automated acceptance tests, this is now less of a concern.


Apple Apostasy

Looks like Why I quit OS X struck a nerve — it is currently down (see web archive).  Also good: Apple has lost the functional high ground.

I particularly like the comment that “It just works” was never completely true.  My experience is that when working with open source codebases, doing so on an Linux operating system comes much closer to “It just works” than doing so on any other.


Rack broke Sinatra

Not rack’s fault, but Sinatra hasn’t released in a while.  Problem has been known since July, and a fix was merged into master in August.  One possible workaround has been posted.  An alternate workaround:

module Rack
  class ShowExceptions
    alias_method :old_pretty, :pretty
    def pretty(*args)
      result = old_pretty(*args)
      def result.join; self; end
      def result.each(&block); block.call(self); end
      result
    end
  end
end

Weblog Software Rewrite Underway

I’ve clearly been neglecting my little spot on the web.

It has gotten so bad that Brendan Eich had to link to a web archive copy of a page of mine.  I must say, however, that it is very ironic and amusing that it is was that particular page.  General outline of my current approach:

...


WHATWG/W3C Collaboration

I’ve been having fun working on the URL Living Standard. All good things must come to an end. Now it is time to spell out a path forward.

...


pegurl.js

pegurl.js is the result of two days worth of work.  While it is undoubtedly buggy and incomplete, it does pass 255 out of 256 tests and that last test is wrong.  For comparison: results from other user agents.

Current work products and future work

...


WHATWG URL vs IETF URI

I’ve been looking into differences between the WHATWG URL Living Standard and the combination of RFC 3986 and RFC 3987.  I’ve come up with an indirect but effective way to identify the differences using urltestdata.txt and addressable.

...


Dreamhost upgrade

Dreamhost upgraded my server to Ubuntu 12.04.  I noticed things breaking in preparation for the move, and things that broke after the move.  If you see something not working correctly, please let me know.


The URL Mess

tl;dr: shipping is a feature; getting the URL feature well-defined should not block HTML5 given the nature of the HTML5 reference to the URL spec.

This is a subject desperately in need of an elevator pitch.  From my  perspective, here are the three top things that need to be understood.

...