intertwingly

iTunes uses a real, draconian XML parser. iTunes the application looks like it contains some case-insensitive processing. For example, this horrible creation seems to function in iTunes. iTunes seems to ignore elements in the iTunes namespace with iNcorrEct cases, but liberally munges RSS elements. But, the parser is still draconian. A document that begins <RsS> and ends with </rss> won’t work.

Oh, and there’s this:

[Macintosh:/Applications/iTunes.app] sayrer% grep -r PodCast-1.0 *
Binary file Contents/MacOS/iTunes matches
[Macintosh:/Applications/iTunes.app] sayrer% grep -r Podcast-1.0 *
[Macintosh:/Applications/iTunes.app] sayrer%

For example, this horrible creation seems to function in iTunes.

But wait!  There’s more!  I’ve devised some more tests ( [link] ).  To sum up what we’ve discovered so far:

• Namespaces are case-insensitive, which is actually a good thing, since the namespace defined in the spec is wrong.  (Holy wellformedness, Batman!  Even my Ultraliberal^H^H^H^H^H^H^H^H^H^H^H^H Universal Feed Parser missed that one.  Quick, to the UnitTestCave!)
• Tag names outside the iTunes namespace are case-insensitive.  Been there, done that.  Look for me at Apachecon this fall; I’ll be the guy wearing a nametag that reads “Hello my name is textinput.”
• iTunes completely ignores HTTP.  Content-type is meaningless, charset doubly so.  Real men use UTF-8.
• All attributes on the enclosure element are optional except url.  On further reflection, this is probably not a bad thing.
• itunes:link is either an element with URL content, a subelement of image with URL content, or an element with rel, type, and href attributes.  The spec says one thing, Apple’s discussion forum says another, and a high-profile feed says a third.  Blog posts and tea leaves...
• channel/itunes:subtitle is displayed properly even when defined parsed as outside the iTunes namespace.  Ditto item/subtitle.
• itunes:duration can also be defined outside without those pesky namespaces.  However, for consistency, it is important to note that iTunes elements that are declared outside the iTunes namespace are case-sensitive.
• Dates can be declared without a weekday, time, or timezone.  June 31st is a legal date, but for interoperability reasons, it is displayed as July 1st.
• If an element contains inline (not escaped) XHTML, only the text content of the last child element is displayed.  This feed displays “now” as its channel description.
• iTunes completely ignores HTTP (part 2).  It will continue to attempt to fetch feeds marked 410 Gone forever.  This reminds me of one of my ex-girlfriends, but that’s a story for another day.
• iTunes completely ignores HTTP (part 3).  It will follow permanent redirects, but it will continue to fetch the old URL forever.  Embrace the impermanence of all things.
• iTunes sends only two HTTP headers: “Accept: */*”, and “User-Agent: iTunes/4.9 (Windows; N)” (presumably platform-specific).  It does not support ETags, Last-Modified, gzip or zlib compression, or RFC 3229.  No pithy tagline, that just sucks.

Y’know, that would be so much more amusing as an acerbic Mark Pilgrim blog post. Nothing against Sam, but a list like that is wasted when buried in the comment section of his blog.

Mark, isn’t it time for an old-fashioned ‘short essay’ section on your homepage?

OK, so I don’t normally beg for links.  In fact, I never do.  It very unbecoming, and — quite frankly — I’m happy with the amount of traffic I attract.  Not too little, and not too much. All that being said, I want to increase... [more]

Trackback from Sam Ruby

details are here: [link]...... [more]

Trackback from Davanum Srinivas' weblog

iTunes has a stupid parser. Read the comments....

So Sam Ruby is pandering for links in a vain effort to tell Apple that iTunes’s RSS support is horribly non-standards compliant.  Of course this effort will fail horrible because Apple doesn’t do anything unless there’s a Radar...

And he asks for a link: here you go....

blech: Insensitive iTunes As well as crapping up the interface, it turns out that the podcast/RSS client in iTunes 4.9 is... lacking. The sadly typically quiet Mark Pilgrim chimes in to the comments of this Sam Ruby post, pointing out the multiple...

Indeed, Mark. All the cool kids are starting online magazines these days.

Sam Ruby wants Apple to address this concern....... [more]

Trackback from Randy Holloway Unfiltered

It seems Sam Ruby and friends have found holes in iTunes. I think it was originally a bunch of small holes, but then it escalated into “Does anybody at Apple have a copy of the XML spec?” [link]...

Sam Ruby wants Apple to address this concern.......

Sam Ruby is calling out Apple for releasing iTunes with a screwy RSS parser and Disney for publishing feeds that fall into the traps set by the parser’s quirks....

Per Sam Ruby’s request, this is an appeal for someone who matters at Apple to please look here. Obviously, there are people at Apple who understand the Net, but for the ones who seem not to, the ones who built the iTunes RSS, here’s how it works:...

Hey Apple, go here and read carefully....... [more]

Trackback from snellspace.com

410 - Gone: Moved to Sam’s comments.

Sam Ruby’s Shameless Pandering prompted me to link to his prior post on the problems with the way iTunes handles XML feeds. The comments in that post explain everything in full detail. If this topic is of any interest to...... [more]

Trackback from Full Speed

Y’know, that would be so much more amusing as an acerbic Mark Pilgrim blog post. Nothing against Sam, but a list like that is wasted when buried in the comment section of his blog.

While I miss Mark’s blog just as much as the next guy, I find it amusing to see him popping up here and there in the comments of blogs that are important to me.

Small update to this comment.  It appears that my missing namespace declaration test was invalid (it was missing the beginning rss tag altogether).  Now corrected, iTunes downloads and parses the feed without error.  It does not find any of the elements in the (undeclared) itunes: namespace, but it does accept the feed, falling back to core RSS elements for display, and merrily downloading the audio file linked in the enclosure/@url attribute.

Because of the undeclared namespace, this feed (despite the URL) is not well-formed XML (libxml2 correctly barfs, "namespace error : Namespace prefix itunes on subtitle is not defined"), but iTunes accepts it anyway.  This, in turn, leads me to revise my previous statement on iTunes' draconianness.  It is NOT a draconian XML parser; it DOES accept a certain class of non-wellformed feeds.

(In iTunes' defense, both Firefox and Universal Feed Parser also think this feed is well-formed.  Firefox displays it as-is, and UFP returns bozo=0.  I have added this as a test case.)

iTunes does not care if the required rss version attribute is missing.

Firefox and Thunderbird 1.0.x are based on a Mozilla branch that’s quite old. On that branch, Expat is configured not to do XML+Namespaces processing. The faulty prefix example is well-formed XML 1.0, so it works. Firefox/Thunderbird 1.1 will have XML+Namespaces processing turned on. The feed failed to load in last night’s Firefox 1.1 nightly (fatal error).

Here’s a great post about iTunes issues with their RSS parser/namespace. (Note the best info is in the comments) Hard to believe that Apple would do this one: iTunes sends only two HTTP headers: “Accept: */*”, and “User-Agent: iTunes/4.9 (Windows;...... [more]

Trackback from Hodson Blog

If the channel element is missing (i.e. channel elements are direct children of rss, item element is direct child of rss), iTunes does not display the title or description of either the channel or the items, BUT it does still download the MP3 file linked from /rss/item/enclosure/@url.

If the item element is a direct child of rss, iTunes displays the channel title and description, does not display any item information, but still downloads the MP3 file linked from /rss/item/enclosure/@url.

If item is the root element, iTunes displays no information and gives an error “There are no playable episodes.”

If the root element is atom instead of rss, which contains channel, which contains item... iTunes displays the channel title and description, does not display any item information, but still downloads the MP3 file linked from /atom/channel/item/enclosure/@url.

If the root element is atom, which contains rss, which contains channel, which contains item, iTunes does not display any channel or item information, but still downloads the MP3 file linked from /atom/rss/channel/item/enclosure/@url.

If the root element is a random name other than atom or rss, iTunes displays the channel information, does not display any item information, and gives an error “There are no playable episodes.”

If the root element is a random name and the channel element is a random name, iTunes displays the channel information, does not display any item information, and gives an error “There are no playable episodes.”

If the enclosure element is a child of channel (instead of a child of item), iTunes displays the channel information, does not display any item information, and gives an error “There are no playable episodes.”

If the rss root element is missing, iTunes displays the item information where the channel information should be, and gives an error “There are no playable episodes.”

If the item element is outside channel but inside a randomly-named element, iTunes displays both channel and item information in the correct places, and downloads the MP3 file linked from /rss/foo/item/enclosure/@url.

If the item element is outside channel but inside 2 randomly-named elements, iTunes displays the channel information, does NOT display the item information, BUT still downloads the MP3 file linked from /rss/foo/bar/item/enclosure/@url.

In summary, it appears that

• iTunes looks for channel information in elements named “title”, “description”, “summary”, “subtitle”, or “itunes:subtitle” (where the itunes: namespace is declared but case-insensitive) within the first child of the root element in the default namespace, regardless of the name of the root element or the name of the first child.
• iTunes looks for item information in elements named “title”, “description”, “subtitle”, or “itunes:subtitle” within elements named “item” that contain an element named “enclosure”, as long as the “item” element is a child of the first child of the root element (all in the default namespace, and regardless of the name of the first child or the root element).
• iTunes looks for audio files in the “url” attribute of an “enclosure” element within an “item” element.  The “item” element can be anywhere in the feed, as long as the root element of the feed is not “item” or “channel”.

Corrections welcome.

Sam Ruby’s Shameless Pandering prompted me to link to his prior post on the problems with the way iTunes handles XML feeds. The comments in that post explain everything in full detail. If this topic is of any interest to you, please link to Sam’s...

The feed failed to load in last night’s Firefox 1.1 nightly (fatal error).

Ah, that explains it.  Yes, the feed is well-formed to a non-namespace-aware XML parser.  However, iTunes appears to be using a namespace-aware XML parser (since the name of the “itunes:” prefix does not matter, as long as the namespace matches case-insensitively to the iTunes namespace).  So iTunes is definitely accepting non-wellformed feeds.

I smell XQuery.

Am I the only one who doesn’t think this is such a big deal? So they wrote a loose parser. So it accepts some things that most other parsers won’t. At least they didn’t add an <itunes:marqee> tag that shows a marqueed text when you play the visualizer...

Jason

From a friend at Apple: apparently, the iTunes folks have seen Mark’s notes. My friend also points out that the service uses Wonder, an Open Source framework for WebObjects. See [link]. So the place to fix the problems are in there somewhere...

Lots of gaping holes in RSS/XML support in Apple’s iTunes. Classic Mark Pilgrim post - errm, in Sam Ruby’s comments ....

We have been waiting for a week for some sort of word from Apple on a RSS tag clarification. Now...... [more]

Trackback from Geek News Central

I second Jacques' request. Bring back the blog, Mark!

Scaling mod_perl (tags: apache mod_perl perl) Sam Ruby: Insensitive iTunes (tags: apple itunes rss) Wage Slaves from 1UP.COM (tags: games)...... [more]

Trackback from orbitalworks

As per Sam Ruby’s shameless request, I’m linking to his post on the problems with iTunes RSS in the hope it will encourage Apple to fix them (scroll through the comments for various horrors). As a competitor, the first thing I need to say is I have...

Am I the only one who doesn’t think this is such a big deal?

Apple is an 800-lb. gorilla in this space (at least until Microsoft releases an RSS-enabled IE in Longhorn).  iTunes is to podcasting as Internet Explorer is to HTML.  RSS interoperability, at least as far as podcasting goes, now means “works with iTunes.”  Thousands of people and companies will begin making podcasts that “work with iTunes,” but unintentionally rely on iTunes quirks (e.g. Disney’s incorrect namespace).  This in turn will affect every developer who wants to consume RSS feeds, and who will be required to emulate all the quirks of iTunes to remain competitive.

Apple has effectively redefined the entire structure of an RSS feed, added multiple core RSS elements, made all RSS elements case-insensitive, made XML namespaces case-insensitive, created a new date format, made several previously required attributes optional, and created a morass of undocumented and poorly-documented extensions... to what was already a pretty messy format to begin with.

Case in point: my Universal Feed Parser, which already has 2751 test cases and is so incredibly liberal that it can parse an ill-formed EBCDIC-encoded RDF feed with regular expressions, will require hundreds of new test cases to cover all the schlock that iTunes accepts.  And I’m one of the lucky ones.

The supreme irony of all this is that I remember Dave Hyatt (Apple Safari developer) bitching and moaning about all the work he had to do to make Safari emulate the buggy, undocumented behavior of Internet Explorer, and how the world would be so much better if only everything used XML and everyone implemented draconian error handling.  Never mind the fact that the vast majority of problems that iTunes creates have nothing to do with XML well-formedness; iTunes doesn’t even require well-formed XML in the first place.  Utopia, it seems, will have to wait another decade.

If you work at Apple, especially if you work on iTunes, please read this post and its comments. If you’re a blogger, please consider linking to the above link....

... [more]

Trackback from Dare Obasanjo aka Carnage4Life

Why Jason Scott’s BBS documentary won’t be on TV Because they’ll take overbbs, tv, video Tweaking PHP 4 on Tiger Using EacceleratorEaccelerator, cite:wfm, osx, php George Bush singing Imagine All we are sayingfun, podcast, politics Stereotyping...

Microsoft recently suggested an extension to an important emerging standard, RSS. They used a grass roots industry conference to discuss their extension before shipping any product which used it and quickly received comprehensive, constructive feedback from a range of smart...... [more]

Trackback from Active Web

RSS Readers: Narrowing Down Your Choices Interesting mix of server- and desktop-based setups. Not sure if iTunes is number 3 because it’s so popular or that’s skewed because it doesn’t support conditional GET (or compression, or...

It’s been reported that iTunes is lax in parsing the RSS extensions that they added to support iTunes....... [more]

Trackback from steve's blog

Mark Pilgrim’s comment looks at ways iTunes isn’t playing nice with RSS specs....

How can I not respond to Sam’s Shameless Pandering for a link to his Insensitive iTunes post? It really is important Apple gets this one right....

The iTunes RSS parser apparently has some interesting issues....

Everyone loves a good pile-on when the behemoth hijaacks a standard. Only this time, the behemoth is Apple who, if......

Mark Pilgrim, commenting on iTunes' RSS handling, wrote: Dates can be declared without a weekday, time, or timezone. June 31st is a legal date, but for interoperability reasons, it is displayed as July 1st. ... iTunes completely ignores HTTP (part...

tags:tags Apple iTunes podcast RSS XML Last Thursday Kevin Marks and I met with some friends on the iTunes team at Apple and talked about tags, podcasts, RSS, XML, spec-writing, validators, and how best to give, watch, and respond to feedback on the...

There are numerous reasons to be nonstandard. For example when they built the world trade center in New York the thing was so huge that the vendor suggested they could have custom light bulbs that screwed in counter clockwise rather than the usual...

Tantek Çelik (t d f), who just a few days ago reached his one-year now-I’m-free-from-Microsoft’s-control milestone, writes about his meeting at Apple with some friends of his on the iTunes (t d f) team. Specifically, they were there to talk......

Lots of activity in the podcasting space at the moment, which hasn’t gone un-noticed by the likes of The Economist, who describe the addition of podcasting support to iTunes thusly Any confusion about the term or the process has not......

Tantek Çelik (t d f), who just a few days ago reached his one-year now-I’m-free-from-Microsoft’s-control milestone, writes about his meeting at Apple with some friends of his on the iTunes (t d f) team. Specifically, they were...

Personally, I usually think that the tech industry has gotten a little too “standards happy”. Moreover, I think that good standards need “extension points” that ensure extended capabillity without sacrificing compatibility. J2EE has a good and bad...

Filed under: Audio, Windows, Macintosh, Blogging, Business, Developer, Internet, Podcasting, TextIt can’t be all sunshine and lollipops in iTunes podcasting heaven, can it? Lest you think I’m the ultimate Apple fanboy, here is a good example of...

Traci: As promised, we [FeedBurner dudes and dudettes] are pleased to announce the immediate availability of iTunes metadata for your podcast. [cut] After a[n] audio post has been created, it must be properly transformed into an RSS2.0 feed...

In 1993, America Online jumped into the Internet with both feet, and the waves took an awful long time to subside. A huge influx of new users discovered Usenet, and thanks to AOL’s poor stewardship of the situation, the newsgroups...... [more]

Trackback from Podcasting

以前とりあげた iTunes の Podcasting ですが、XML のコミュニティではすこぶる不評のようです。というのも、iTunes 内で RSS を解釈するパーサーが大文字小文字を区別していないとか。 まさか Apple がゼロから XML パーサーを実装しているわけでも無いだろうに、なんでやねんと思うと同時に、この人たちはいったいどんな使い方をしたら、そんなことに気付くんだろう? という疑問も。まあ、世の中にはすごい人がいるものですね。...

I was disappointed to discover that while it has no problem interpreting podcasts distributed as RSS 2.0 with <enclosure> tags, iTunes completely ignores Atom feeds with <link rel=”enclosure”> elements.  In fact, it’s pr...... [more]

Trackback from dsandler.org

Hmm ... “The impertinence of all things” ... has a nice ring to it. Inspired by one of Mark Pilgrim’s comments on Sam Ruby’s entries (here and here) about Apple’s Podcast specifications. Or should I spell that "aPplE"? I guess they won’t care....

Everyone with a syndication feed should be subscribed to Sam Ruby, especially those high profile companies creating non-standard feeds. If you don’t do it to watch your feed’s back, then you simply must do it to catch the pithy Mark Pilgrim...

Sam Ruby’s Shameless Pandering prompted me to link to his prior post on the problems with the way iTunes handles XML feeds. The comments in that post explain everything in full detail. If this topic is of any interest to......

Passata l’overdose di blogging fatta con Blogo di cui parlerò un’altra volta, mi sto accorgendo di quanto siano noiosi, ripetitivi e spesso approssimativi buona parte dei blog. Il mio livello di tolleranza ha raggiunto il limite per i pettegolezzi e...

Everyone with a syndication feed should be subscribed to Sam Ruby, especially those high profile companies creating non-standard feeds. If you don’t do it to watch your feed’s back, then you simply must do it to catch the pithy Mark Pilgrim...

Sam Ruby asks for linking to the discussion of Apple’s RSS extensions in his blog. It’s a worthwile read on how to (and especially how not to) extend existing XML formats. The topic is quite interesting. I’d be interested in a more...

It’s been reported that iTunes is lax in parsing the RSS extensions that they added to support iTunes. In a nutshell, the way iTunes finds the nodes it needs in an RSS document is apparently by scanning through the document doing a case-insensitive...

Does anyone have a working RSS file that iTunes will accept and parse that I can look at?  I’m having trouble working thru all the non-standard bugs and know that if I could look at a fully working RSS file I could then get mine working.  Jamesfa@autobytel.com - Thanks!

James, the Sample in this document is a good start, with a few exceptions:

• The namespace is incorrect, use http://www.itunes.com/DTDs/Podcast-1.0.dtd instead.
• The line containing <category text="Food"> should actually read <category text="Food"/> (note the addition of a slash).
• Similarly, the line containing <category text="Politics"> should actually read <category text="Politics"/>.

Once you have a feed, you can use the online Feed Validator to check conformance to the published specifications as we best understand them.  It might make sense to check back periodically as we aggressively update the validator as we find out more information from Apple.

Telegraph | News | Soldiers forced to shout ‘bang’ as the Army runs out of ammunition "Why should soldiers who are being sent to Iraq, where their lives will be endangered, be forced to shout ‘bang’ in training because someone in the Ministry...

links for 2005-08-01 Fresno Famous - Flowing with Famous Podcast A very interesting and well done little podcast for Fresno California (I assume) :) (tags: fresno california podcast) RSS: Dave winer points out some issues with Apple’s RSS extensions...

[link]

Apple has created a syndiation-dev mailing list specifically to have an open discussion about the use of syndication related technologies in Apple products.

Referrer inspired communication is nifty. [link].org/ brought me this gem regarding semantics of HTTP’s PUT vs. POST. My recent encounter with this while digging about REST is the XML.com article “How to create a REST protocol”...

I spent Friday and Saturday in Greensboro, NC at the ConvergeSouth 2005 conference. I had a great time meeting fellow NC bloggers and attending talks on community building, blogging and journalism, collaboration, blog tools and podcasting. The next...

iTunes 6.0 now supports Atom 1.0 feeds.

Test case: [link]

iTunes 6.0 now supports Atom 1.0 feeds.

Test case: [link]

Things that work:

• feed/title displayed as podcast channel name
• feed/subtitle displayed as podcast channel description
• entry/title displayed as podcast item name
• entry/summary displayed as podcast item description
• entry/published displayed as release date
• entry/link[@rel='enclosure']/@href downloaded as enclosure

Thanks to the Apple dev team for supporting open standards!

In a posting in response to my comments on iTunes' botched one-click subscribe, Peter van Dijk brings in an ecological metaphor: I think most developers feel they have better things to do than to get Apple to clean up their act. Me included. We’re...

so I just downloaded itunes, something i’ve never bothered with before because a) I never buy downloaded music and b) i don’t have an ipod. even when i reinstalled quicktime, i installed the previous version, as the newest comes packages...

Depressing as usual: "To sum up, the “photocasting” feature centers around a single undocumented extension element in a namespace that doesn’t need to be declared. iPhoto 6 doesn’t understand the first thing about HTTP, the...

Everyone with a syndication feed should be subscribed to Sam Ruby, especially those high profile companies creating non-standard feeds. If you don’t do it to watch your feed’s back, then you simply must do it to catch the pithy Mark Pilgrim...

After a fairly disastrous start last summer, Apple’s podcasting specifications are improving. Let’s hope the same happens with photocasting....

a list of detailed changes please Stop doing this. Stop doing this. Stop doing this. Stop doing this. Stop doing this. Stop doing this. Fix this. Fix this. (At least the part towards the end about not shitting all over the Net.) Fix this. Let

Oh come on, Dewitt, you’re not even trying! Niall hands you the perfect opportunity to diss Apple, and the best link you can come up with is a weak-ass Winer rant? I can think of plenty of better links for a phrase like "Apple’s casual relationship...

Everyone with a syndication feed should be subscribed to Sam Ruby, especially those high profile companies creating non-standard feeds. If you don’t do it to watch your feed’s back, then you simply must do it to catch the pithy Mark Pilgrim...

The Wall Street Journal piece on Apple’s iPhone arrangement with Cingular is a worthy read. One section of this article shows a disconnect from reality though: Mr. Jobs once referred to telecom operators as “orifices” that other...

Anne van Kesteren : Insensitive iTunes - Mark Pilgrim is back. kayodeok : Sam Ruby: Insensitive iTunes - Mark Pilgrim: it appears that iTunes uses a real, draconian, namespace-aware XML parser... except that namespaces are case-insensitive...

“iTunes 6.0 now supports Atom 1.0 feeds.”...