As a learning exercise, I tried converting the Universal Feed Parser to Python 3.0. I picked it because it is a relatively self contained code base that I am familiar with, one that is actively in use, and one that has seen the wear and tear of dealing with compatibility (and the need to monkeypatch the occasional bug) of a number of Python releases.
$ svn co http://svn.python.org/projects/sandbox/trunk/2to3/
$ cd 2to3
$ python refactor.py -w ../feedparser/feedparser.py > feedparser.2to3.diff
root: Generating grammar tables from /home/rubys/svn/2to3/PatternGrammar.txt
root: Writing grammar tables to /home/rubys/svn/2to3/PatternGrammar.pickle
../feedparser/feedparser.py: At line 3091: You should use a for loop here
RefactoringTool: Files that were modified:
$ python refactor.py -w ../feedparser/feedparsertest.py > feedparsertest.2to3.diff
RefactoringTool: Files that were modified:
A few manualchanges later, and 91% of the tests pass. I’m confident that with a little more work, I could quickly get that to 99%, perhaps even to 100%.
Python 3.0 feels very natural and comfortable, at least to this Python programmer. The language feels clean and new again. No more “new-style” classes. Things that should have been iteraters all along now are. Python 3.0 is full of those kind of small changes.
No bugs were found in the language, though some minor issues (noted below) were found in various parts of the runtime.
The 2to3 conversion, even at this first alpha sandbox state, was painless and efficient. It also didn’t reduce the readability or break the functionality of the code produced.
The places where manual attention is required did generally seem to be places where human attention is required.
The Unicode change (while *VERY* welcome) is going to hit people hard. While the UFP code has greater than its fair share of such code, the fact that people will no longer simply be able to open(file).read() unless that file is utf-8 is going to be a big shock.
Some of the python3 libraries don’t seem to have internalized the Unicode changes yet. base64.decodestring can only handle bytes, not characters (why?). More troublesome to me is that I couldn't get xml.sax.xmlreader.InputSource to work with a io.BytesIO (a StringIO work-alike for bytes), but instead only seemed to work with Characters — something that is at odds with proper handling of XML. However, I do realize that this is an alpha, and fully believe that these issues will be worked out.
The test code for UFP relies on eval, and some portion of those strings — ones that can never be automatically handled by a 2to3 migration tool — rely on Python 2.x specific syntax. Ultimately it might be worth considering introducing an python2 module with functions like eval that can be used to ease migration.
The places that the 2to3 migration tool can’t currently handle, and perhaps never will be able to handle, will often require somebody who has an understanding and a history with the code base. Such people aren’t always available. I’m not sure what can be done about that.
3.0 will only get better. I’m highly confident that the issues listed above will be either eliminated or mitigated. The one question that this exercise raised for me that I can’t seem to shake is this: what is the role of 2.6? If it introduces new stuff that makes the migration easier, I can’t see libraries like the feedparser making much use of them until it is 2.5 and 2.4 and even earlier versions of Pythons are pretty much a thing of the past. I’m sure I’m missing something here, but I just don’t see it.
Last Friday, Guido van Rossum released the first alpha version of Python 3.0 aka Python 3000 . Judging by the changelog , the Python developers got rid of a number of annoyances and inconsistencies in order to make the language more cleaner...
I probably should mention that as ASCII is a proper subset of utf-8, open(file).read() will likely work for most of your test data and wait to fail until after you deploy, probably as a result of somebody copy/pasting so-called “smart quotes”.
Also, open(file,"rb").read() will return raw bytes.
Simon Willison : Sam Ruby: 2to3 - Sam Ruby: 2to3. Sam’s report on an attempt to port the Universal Feed Parser to Python 3.0. The 2to3 tool does most of the work, but it seems the unicode changes can be pretty tricky....
Scopro dal blog di Guido van Rossum che ieri è stata rilasciata la 3.0a1 di Python – famoso e amato linguaggio di programmazione multiparadigma*. Su python.org (sito ufficiale) trovate la pagina della release. E’ inoltre disponibile...
The one question that this exercise raised for me that I can’t seem to shake is this: what is the role of 2.6?
Sure, it’s about migration. The process description is such that all code shall be adapted in Python 2.6 with appropriate hints and none in Python 3.0. Together with these hints the conversion process shall become fully automized. I didn’t examine 2to3 yet and I’m not sure how the the user provides hints to the conversion tool. It might even be that Python 2.6 introduces new language syntax for this purpose only.
The process description is such that all code shall be adapted in Python 2.6 with appropriate hints and none in Python 3.0.
This works well for top level applications, but not so well for library developers for which the library is not part of Python itself. Specifically, it would require a number of libraries I depend upon (e.g., httplib2, feedparser, html5lib) to stop supporting Python 2.5.
Sam Ruby tryied to port the Univesal Feed Parser to Py3k. 91% of the tests pass, what is quite nice. UFP is a great piece of code, and one of the first things I would miss. PyGTK people started talking about the migration - this will be very...
Today I’m announcing I’m throwing my hat in the ring to port feedparser to Python 3. There’s a ticket open regarding this at the feedparser bug tracker, but the person who’s working on porting it appears to be writing Python 3 code instead of...