Yesterday, Python 2.5 was released. That means that it is time to find out how good the test coverage is on the various tools I care about. Within the past 24 hours, I made fixes to the Feed Validator, the Universal Feed Parser, and Venus. I also prepared this patch for Beautiful Soup — note: there still is one test that fails on 3.2 that I don’t understand, but that can wait for another day.
The change was typical of what I found. The code is meant to compare the number of children that have the same name against the constant 10. __cmp__ was a member function defined on the string and unicode classes, but that method was apparently removed in favor of defining __eq__. For most usages, this change is transparent, but not this one. And I can’t change to simply use __eq__ as that method isn’t defined on strings in Python 2.4. So, I used list comprehensions instead. Sometimes it is a good thing when there are more than one way to do things.
I’m confident that the test coverage isn’t 100%, and in fact I am currently writing up more test cases for the Feed Parser; but even so, passing the tests that have been written to date, and in the process only finding minor issues does increase my confidence.
But that’s not the full story. The changes to sgmllib in particular could have been considerably more painful had I not gotten involved before the release.
Sam — I’m not sure if you’ve seen any posts on Planet Python about the Pybots project, which aims to do run automated tests for Python projects upon each and every checkin that happens in the Python trunk (2.6) and in the 2.5 branch.
If you’d like to add a buildbot buildslave to the Pybots mix, or even if you’d like some of your projects tested by people who already donated buildslaves, please go to pybots.org, poke around a bit, and send a message to the mailing list).
Ha, I see Grig beat me to the punch, but I was just looking into feedvalidator to see how it can be tested (I am already running a buildslave for the pybots community project). Sam, if you’d like me to run the tests for feedvalidator and feedparser and venus in my buildslave, it would be nice if you could contact me and tell me what the easiest way is to run all of the tests for each package. It would be nice if there’s a way to run them from Python, but of course just os.system() will work if it’s all shell code (which seems how the tests are put together in feedvalidator).
I figure maybe performance is not an issue here, but is there a reason you have not used operator.eq() as the filter function? That would avoid building a new list...
PyBots looks cool! All the projects I mentioned have automated, Python based regression tests. In the case of the Feed Validator, the test in question can be found in src/validtest.py. My one requirement is that regression tests not be done against some prior snapshot of the code being tested, but against the latest. After all, should the Feed Validator (which is currently being run on 2.4) make use of a variable named with or as, I’d simply correct that in the latest SVN to make later transitioning to 2.6 easier.
Should read 2.4 I believe
Fixed. Thanks!
is there a reason you have not used operator.eq() as the filter function?
I actually needed a bound function (i.e., one with only one remaining parameter)
Regression Testing for Python 2.5 - Dynamic typing, seperate compilation, backwards compatibility, test coverage - spin your own cautionary tale out of this one[Add a comment]...
The more I look at generational changes, the more I am astonished by the ability of people to simultaneously (1) accept the rage of change in the past, and (2) presume that what is now ubiquitous will remain so. First an anecdote similar to Jim’...
[more]
A rule that I’ve learned over the years of Python development is:
If you’re using a double-underscore attribute of something, then raise the yellow flag. There is almost always a better approach.
For example, in your original code, rather than using __cmp__, I would have broken out a helper function and did a simple “==” test like in your new code. Or even used cmp() in the helper if that was truly important (which it wasn’t, as cmp()-like results were used as a hack into the filter() function).
In short, the use of __cmp__ instead of cmp() was the root problem. The yellow flag on __cmp__ would have caused you to reexamine and rebuild, which would have then provided a more stable implementation.