It’s just data

Intermittent IE failures

Ziv Caspi: None of the pages on your site can be viewed by IE (except for the Atom feed).

This is the second report I’ve gotten on this.

What makes this even more frustrating is that I know that my site can be viewed by IE, as I get a slow but steady bit of AdSense revenue from IE users, and I’m not being able to reproduce the problem myself.  Via email, Ziv now tells me that he can’t reproduce the problem either.

What would be helpful would be if somebody can find a way to reliably reproduce the problem.  What also would be helpful (actually, probably even more so) would be if somebody could report the values from HTTP_ACCEPT and HTTP_USER_AGENT from this page on a machine where the failure is reproducible.

Background

I want to continue to serve small, embedded, and resizable SVG images to people who use browser that support such things.  This includes Gecko based browsers (like Firefox), Opera, and recent versions of WebKit (the engine inside Safari).  Such browsers will only display such content if the page is served as application/xhtml+xml.  IE doesn’t currently support SVG, nor will it render any page served as application/xhtml+xml, at least not without plug-ins such as MozzIE.

Being an optimist, I don’t want to blacklist browsers like IE and Safari, I want to make a check based on capabilities, and that’s what the HTTP accept header is meant for.  Unfortunately, the accept header that MSIE sends is less that fully helpful:

image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/x-shockwave-flash, application/x-silverlight, */*

Note that application/xhtml+xml isn’t listed.  Nor is text/html.  What is listed is a number of image formats (Firefox, by contrast lists only one image format: image/png), a number of proprietary formats, and the catch-all */*.

*/* may be technically accurate, but completely misses the point.  By way of contrast, Google’s crawler sends out an accept header which simply states */*, and they mean it.

So, for some time now, and in deference to IE users, I have looked for image/gif and used it as a proxy for saying “I support text/html”.  Not perfect, but it worked for me.

But apparently not consistently for everybody.  I’m looking for better suggestions.


I’m sure I reported this to you once before. What happens (at least for me) is that IE7 sends */* as its accept header when you open a link in a new tab or new window (which causes your site to break). I suspect people don’t realise that’s what they’re doing so it’s seems like an intermittent problem. These are the values I get back from your test page:

HTTP_ACCEPT */*
HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)

I just checked IE6 on my XP machine and it doesn’t have this problem.

Posted by James Holderness at

If this helps, I noticed it happens only on index pages, not on single posts. This should explain adsense revenue (which usually comes from single-post pages, in my experience)
Headers:

HTTP_ACCEPT */*
HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.1)


Posted by Giacomo at

Sam, I think you may have your algorithm doing it backwards. Instead of defaulting to application/xhtml+xml and just reverting to text/html if you find preference for it, I think it’s better to default to text/html and then only emit application/xhtml+xml if you find explicit mention of it in the Accept header.

All browsers I know of that supports XHTML states this clearly in their Accept header, so you shouldn’t get any false negatives afaik, and since Internet Explorer doesn’t mention either application/xhtml+xml nor text/html, it won’t get a false positive.

If you want to be really advanced, you could of course pay attention to the q= values; that is if browsers are so weirdly set up that they support XHTML but prefer it served as HTML. I wouldn’t invest much time into getting that up and working, though.

Posted by Asbjørn Ulsberg at

they support XHTML but prefer it served as HTML

This was true recently (but is now fixed?) for Gran Paradiso - the Mozilla devs decided that until recently their rendering of text/html was better than their rendering of application/xhtml+xml as various things like incremental rendering were broken in XML.

Posted by Robin at

The problem was intermittent for me as well. But when it did occur, my headers were


HTTP_ACCEPT */*
HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Tablet PC 1.7; .NET CLR 1.0.3705; .NET CLR 1.1.4322; InfoPath.1; .NET CLR 2.0.50727)



Posted by Dare Obasanjo at

IE doesn’t currently [...] render any page served as application/xhtml+xml, at least not without plug-ins such as MozzIE.

In my experience IE7 (I didn’t test earlier versions) will happily render pages served as “application/xhtml+xml” as long as the filename ends in “.html”. That said, IE7 renders such pages as HTML rather than XHTML, so it’s more of a quirk than a usable feature.

Posted by scott lewis at

Remove */* from the Accept header.

Let X be the set of all media ranges in the Accept header that match application/xhtml+xml. Is X empty? If so, send HTML.

Let H be the set of all media ranges in the Accept header that match text/html. Is H empty? If so, send XHTML.

Let Qx be the maximum q parameter value in set X; let Qh be the maximum q parameter value in set H. If Qh > Qx then send HTML; otherwise, send XHTML.

This will send XHTML to Safari 3, Opera 9, and Firefox 2, and HTML to IE and Google. If you want to send XHTML to Google and HTML to IE then you need to do User-Agent sniffing; as James said, IE will often send “Accept: */*” just like Googlebot does:

Start|Run, "iexplore [link].cgi"

...
HTTP_ACCEPT */*
HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
...

Posted by Brian at

HTTP_ACCEPT */*
HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)

Only happens for me on THIS page, not on the homepage or any other blog page.

Posted by Ryan Adams at

I have made the following change to my .htaccess file:

# MSIE accomodation: open in new tab/window
RewriteCond %{HTTP_ACCEPT} ^\*/\*$
RewriteCond %{HTTP_USER_AGENT} MSIE
RewriteRule ^([^.]*|.*\.html)$ - [T=text/html;charset=utf-8]

Hopefully this will help.  A few notes:

Posted by Sam Ruby at

On my blog, I’m using an algorithm similar to what Brian suggests.

Using a Ruby port of Joe Gregorio’s mimeparse, I test the preferredness of “text/html”, “application/xhtml+xml” and “imaginary/fake”. Send real XHTML if it beats both “text/html” and the mimetype that nobody will ever put in their Accept header, HTML otherwise.

Posted by Brendan Taylor at

I started doing “Content Negotiation” on all my sites some time ago and worked hard to stamp out edge cases and follow the HTTP Specification. My algorithm works like this:

There are two methods I use for "best guess": first specificity, if that fails default to one of the types (HTML).

Specificity works by choosing the media type with highest specificity. If the accept header sends ‘text/html, */*’ both XHTML and HTML media types have a q value of 1, however, the HTML media type has a higher specificity so we choose that one.

If both HTML and XHTML media types have the same specificity (e.g. '*/*') then default to one or the other. This will likely be HTML.

I also use an override option for XHTML. That is, if an accept header shows that HTML and XHTML have the same quality value, same specificity, but XHTML media is stated explicitly then choose XHTML (e.g. text/html, application/xhtml+xml). Specifically done for Opera, but is a good way of handling cases where you want to default to HTML but prefer XHTML if supported.

All of the recent work and refinements I’ve done are in C# for ASP.Net, but I do have a (perhaps outdated) PHP script ([link]) some documentation can be found at [link].

Posted by Jonathan Porter at

If both HTML and XHTML media types have the same specificity (e.g. '*/*') then default to one or the other. This will likely be HTML.

If the client indicates no preference, the server should make a decision based on the characteristics of the page.  If the server produces both XHTML and HTML for no reason, picking one at random would make sense.  If the server produces both XHTML and HTML, it probably does so for a reason.  Do you prefer robustness or quick feedback loops in the case of an error?  Do you make use of features such as Ruby, SVG, or MathML which (as of this moment) are only officially supported in XHTML?

Posted by Sam Ruby at

here’s my experience w/ MSIE : 
this first time you visit a page, the following is in the accept header (from your test page):
HTTP_ACCEPT image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-silverlight, */*
HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; InfoPath.1)

pressing the refresh button sends this:
HTTP_ACCEPT */*
HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; InfoPath.1)

pressing ctrl-f5 sends the */* version of HTTP_ACCEPT.

i don’t know why MSIE does this, but it’s consistent, AFAIK.

stinks, eh?

Posted by mike amundsen at

In my case I only negotiate between HTML 4.01 Strict and XHTML 1.0 Strict. So the difference is essentially none. The purpose of my approach was to do everything “the HTTP way”. When that fails use heuristics.

I said default to HTML simply because real world browser experience shows that browsers that support XHTML state that they do specifically, but I see your point, Google, for instance, indexes all my pages as HTML and the WC3 validator validates the HTML version.

One approach is to provided the XHTML version with more weight. This is what the Apache content negotiation mod does. It is also in my PHP code. The XHTML_RELATIVE_SOURCE_QUALITY and HTML_RELATIVE_SOURCE_QUALITY variables are multiplied by the resulting quality values for each type. They are both set to 1 by default, but the XHTML value could be given a higher value then HTML. The downside being that the quality values are fairly arbitrary.

The only other option I could come up with is using the user-agent header as part of thecontent negotiation scheme if the accept header fails to pick a “winner”, however, that comes with its own problems and still doesn’t guarantee the “best” representation is set in all cases.

Posted by Jonathan Porter at

Re: University Seeks Community Input in Enhancing its Main Web Homepage

[quote Joseph Recht ref=22557:22723]The website needs to be fully standards compliant (XHTML 1.0 Strict, please!)[/quote] It’s [url=http://www.hixie.ch/advocacy/xhtml]impossible to be XHTML Strict compliant[/url] (well,...

Excerpt from General Discussion at

MSIE seems to mostly work right for me, except maybe the ultra unstable wine+ies4linux combo on my box.

OTOH I have a new report, fairly interestingly looks like a bug in either the i18n javascript or firefox 2.0.0.6

In a machine with Windows XP standard, in Spanish, a Firefox set up to prefer English spins forever on a blank page, upto the moment we say “stop script” to the prompt, and then it shows the page with the dates in Spanish

The same machine works like a charm with MSIE 7.0 (now), showing the dates in English as configured. I’ve just seen it (non)working, and I have had no time to look at the sources for the problem. Just FYI.

Any clue? (I know it looks improbable, but always close people do have improbable configurations in this world of choices. :)

Posted by Santiago Gala at

Any clue?

If you view source on this page, you will see dates in GMT.  But on my screen, they show up in EDT. The script to do the conversion is localize_dates.js.  Presumably it has a bug in it.

Can you copy/paste the date you saw — the one in Spanish — here so I can debug it further?

Posted by Sam Ruby at

More data points re: Firefox and funky locales:



Posted by Santiago Gala at

More data points re: Firefox and funky locales:



Posted by Santiago Gala at

I’m testing right now, and MSIE 6 is drawing the NavBar below the pages, not at the right. I’m not sure if it is a known problem. Also, the timestamp of the posts is at the left of the page.

MSIE 7 draws it much better, but the previous problem kicks in, at least for some boxes.

Posted by Santiago Gala at

Grading a Website's Markup

Less Than Perfect It’s difficult to compose a prescriptive list of all the issues a “perfect site” must satisfy. So,......

Excerpt from Jeremy Smith's blog at

Hello,
I am facing problem regarding site working properly on internet explorer 6.0 but not on mozilla. i shall be very thankful to you if you would be able to provide me the solution to get rid of this problem.

Posted by Puneet Kumar Chauhan at

IE6 Accept Header is Faulty and Makes format.any Suck

I’ve been using the awesome acts_as_authenticated plugin as the basis for user login and authorization. Its access_denied method takes advantage of the ActionController::MimeResponds.any method which will be invoked on a login_required before...

Excerpt from Gem Install That at

jQuery, jRails and the Accept header

I included jQuery by installing the jRails plugin, which seems nice because you can continue using rails’ ajax helpers. But when I tried to implement a more special functionality by using the $.ajax function it proved impossible to set the...

Excerpt from Moserei at

jQuery, jRails and the Accept header

I included jQuery by installing the jRails plugin, which seems nice because you can continue using rails’ ajax helpers. But when I tried to implement a more special functionality by using the $.ajax function it proved impossible to set the Accept...

Excerpt from Moserei at

Add your comment