Need for More System Tests
Rails 2.3.6, 2.3.7, 2.3.8, all in less than 48 hours. My tests played a role in getting rails back on course. Overall, I continue to believe that it is much more important for a project to be responsive than correct. Ideally, however, these problems would have been noted before shipping. At least two corrections make sense in response:
- The problems fixed in 2.3.7 were real problems, just ones that don’t affect the readers of Agile Web Development with Rails, edition 3. Specifically the rails_xss plugin, which was introduced both to address a specific security issue and to aid in the transition to Rails 3, caused problems for HAML. More continuous testing of both this plugin and HAML are in order.
- While Rails has an impressive number of tests, nothing replaces actual usage. My tests passed on 2.3.6, and indicated at least two problems of the problems that needed to be fixed in 2.3.7, and pass again on 2.3.8. We need more people to do similar things. And we need the Rails team to look for indications that these types of test pass before shipping. I’ll be discussing this at RailsConf in just over two weeks.