intertwingly

Jacques Distler: if anyone tells you: “i18n is easy, just use utf-8!” … go ahead and smack them.

Luckily, I’m outside of arms reach.  You see, my weblog is 100% valid XHTML 1.1, encoded as utf-8.

Truth be told, however, it also would be considered as 100% valid XHTML 1.1, encoded as iso-8859-1 (roman), iso-8859-5 (cyrillic), win-1252 (Microsoft), or macroman (Apple).

In fact, it would also be 100% valid if it were declared as encoded as us-ascii.

So, I’m not actually disagreeing with Jacques, and therefore probably could avoid a smack.  I would however, quibble with his first line:

i18n is hard. Don’t let anyone tell you any different.

Certain versions of certain tools don’t handle utf-8 well: that much I agree with.  I would also add that finding the right combination for a given configuration and keeping it working through upgrades is a bit of an effort.

But i18n != utf-8.  You can i18n with us-ascii just fine.  Just use numeric entities.  In my Python implementation, the logic to do this is a bit spread out, but I have build a more compact Ruby version.

Moral of the story, don’t convert to utf-8 in a plugin unless you are certain that every link in the chain can handle utf-8 properly.  If you feel that you must convert to utf-8, then is best to do it as some sort of post-processing filter after all the other logic has taken place.

But, for me, numeric entities are just fine.