Planet Intertwingly

Holiday goodies for your Mac

When your Mac is central to your home, it only seems natural to decorate it for the holidays. Historically, the Mac has enjoyed lots of creative holiday-themed enhancements, and Mac OS X continues the tradition.

Yet Another Planet Refactoring

A little over a month ago, I outlined how I would like to see the feed parser reorganized.  I’ve now put a little meat on the bones, in the form of running code.  Not just for the feed parser, but also for Planet.  I also did it all in Ruby, so I named this little experiment Mars.  Warning: this version is 0.0.1.  It just barely runs end-to-end.  Feed it real data, and it will choke on some of it.  But it can now produce partial results.

Inventory:

config
To keep things compatible, I ported the parsing logic from Python’s ConfigParser to Ruby.  Once parsed, the results are accessed as a Hash.  Eventually, I’ll provide Planet specific logic, like defaults, in this module
fido
Fido fetches (get it?) feeds.  It caches, follows redirects, times out, handles HTTP status codes, and compression/zip.  It also distributes work across multiple threads.
xmlparser
This module will use one of four XML parsers, and return the result as a REXML document.  If installed, it will use the fast and standards compliant expat or libxml2 parsers.  If neither are installed, it will use the slower and less compliant (to this day, it still parses <a>&</a> without error).  In all REXML isn’t too bad... as long as you depend on it for serialization or deserialization or XPath or expect quick turnaround on bug fixes or responses on their mailing list.  In the event the chosen parser fails to parse the document, the HTML5lib liberal XML parser will be used, and a bozo flag will be set on the document itself.
Transmogrify
With RSS, there often are several ways to express the same concept (shades of TMTOWTDI?).  Atom aspires to Python’s philosophy of There should be one—and preferably only one—obvious way to do it.  This module is clearly opinionated software in that it will transmogrify feeds which use less obvious constructs into more obvious ones.
sift
Sift will filter out impurities and break down HTML into elements than can be iterated over.
forgiving_uri
This was imported from Bob Aman’s GentleCMS.  I need to look into Addressible (also from Bob Aman) to see if it is a better alternative.
spider
Spider orchestrates the retrieval and sifting of feeds, breaks the results into a set of entries, adds in source information, and caches the results.
splice
Splice will select the latest entries and produce a feed, and then process this information using a user supplied list of templates.  At the moment, the only templating language supported is XSLT.
style
If libxslt is installed, it will be used, otherwise an attempt is made to shell out to xsltproc.
harvest
Not yet integrated into the Mars proper, but used to drive the testing (and therefore the development) to date, this function dynamically constructs dictionaries from an Atom document.  This will undoubtedly be useful for other templating languages.  As everything is constructed dymanically, multiple date formats, mutiple serialization formats (think: dropping superfluous quotes on attributes and pesky things like explit paragraph end tags for HTML5 purists), and multiple aliases for any given element are no problem.

Also provided

planet
A small main program to drive the execution.
reconstitute
A demo program which will enable you to see what the parsed, transmogrified, and sifted version of any given feed looks like
test/feedparser
Runs a small portion (currently wellformed/(atom10|rss)/*.xml) of the feedparser test suite.  Check the comments to see what is not yet supported (mostly elements like cloud and textInput

All in all, I’m pleased with how compact this code is.  If anybody wants to join in on the fun, it is available as a bzr repository and there are plenty of test cases ready to be ported.

FeedSync now covered by the Open Specification Promise

I've been talking to more people about FeedSync since I posted the v1 specification earlier this month. One common question people ask is about licensing terms for the spec, and what our thoughts are about FeedSync implementations.

We've always made it very clear that our goal is to have lots of independent implementations of FeedSync, and that we want to enable royalty-free use of the specification. The spec itself is released under a Creative Commons license; today we're also making the spec available under the Microsoft Open Specification Promise. The full details are in this post on the FeedSync news blog. The short version is that the OSP makes life very easy for implementers. By placing the FeedSync spec under the OSP we're saying even more clearly that developers have royalty free use of the spec, now and in the future.

If you're new to the Open Specification Promise, they have a great page with an FAQ that explains the program in detail: http://www.microsoft.com/interop/osp/default.aspx. The Feedback from Representatives of the Community link shows the response from several people in the open source community; it's been very positive. You can also take a look at our FeedSync FAQ for some specific questions and answers about FeedSync and OSP.

It's great to have an existing program like OSP that is such a good fit for FeedSync, and I've really appreciated the support from the folks in the Microsoft Interop group in bringing the FeedSync specification under the Open Specification Promise.

Back To The Future: IBM's Five In Five

Our prognosticators have unveiled the second annual “IBM Next Five in Five” innovations that they believe have the potential to change way people work, live and play over the next five years.

First on the list: It’s easy being green after all, and you can save money doing it. There will be a range of “smart energy” technologies emerging to make it easier for you to manage your personal “carbon footprint.”

Hummers and unrechargeable batteries not included.

And no Kermit jokes, please.

Second, the way you drive will be completely different. In the next five years, there is a coming “wave of connectivity” between cars and the road that will change the way you drive and keep you more safer.

Forgot that cop in the traffic box: we’re talking intelligent agents and sensors along the highway that make it easier to navigate and avoid that idiot in the El Camino texting while driving.

“Helloooo, could you watch the road, please?!”

Third, you are what you eat, so you will know what you eat. Advancements in computer software and wireless radio sensor technologies (fancy phrase for RFID) are going to give you much more detailed information about the food you are buying and eating.

Do you really want to know the provenence of that “WhoppingBigBurgeraSaurus?” Really?

Fourth, your cell phone will be your wallet, ticket broker, concierge, bank, shopping buddy, and all around BFF. Everybody needs a Best Friend Forever, and why not your mobile device?

Hey, have you ever seen someone using their iPhone? Their Blackberry? Like really seen them?

The phrase “public display of mobile phone affection” comes to mind.

I hereby do not exonerate myself from said mobilerati.

And finally, five, doctors will get enhanced “super-senses” to better diagnose and treat you. Amen, brother. I have to say, I’ve had some pretty good medical experiences recently (knock on wood), between my sleep apnea surgery and my recent LASIK experience. I’ve never had better vision or slept better in my life.

But, according to our five in five projections, doctors will soon be able to see, hear, and understand our medical records in new ways: X-ray like vision to view medical images, super sensitive audio to hear the tiniest audio clue in your heart beat.

That’s all well and good, but will anybody be able to read their prescriptions, or get to your examination room before an hour passes buy? Now that would be innovation.

You can check out the video news release of the "Five in Five" here.

Technorati Tags: , ,

Technorati Considered Harmful


[First published on GOTO XO as Technorati considered harmful on December 19, 2007.]

I just came across Matt Mullenweg’s post 4 Ways to Get Banned from the Technorati Top 100 courtesy of Sam Ruby’s planet.intertwingly.net.

This came as news to me, and sad news, too. While Technorati is not itself open-source it does aim to provide a service to many people, including open-source programmers such as myself.

If Matt’s charges are true, and I see no reason to doubt them, then Technorati is failing to live up to a fundamental credo of open-source: Give Credit Where Credit Is Due.

Though I have been listing my Technorati Rank and Authority numbers on this blog and my personal blog, I will delete them forthwith.

My first blog, The Wayward Word Press (TWWP), has achieved quite a high ranking, due to the over six hundred posts I have published in the last fifteen months and the links to it from others that are measured by Technorati.

My newest blog, and the one closes to my heart these days, is this one, GOTO XO.

I have been republishing my XO posts to TWWP to take advantage of of its higher rating, in the hopes that will lead more viewers to the XO blog itself. I will continue to monitor the Technorati rankings privately, and will suspend the dual publishing when GOTOXO’s rank is sufficiently high. After all, Rank doth have its privileges, even if not fairly measured.

New privacy tips video series



In order to give you the best possible information about the privacy settings for our products, we asked the engineers and product managers who actually designed them to explain how they work in a series of new videos we released today on our YouTube Privacy Channel. These videos feature Googlers sharing privacy tips, like how to use Google Chat’s “Off the Record” feature, how to limit the number of people who can view your Picasa photos, how to unlist your phone number from Google search results, and how to make the details of your Google Calendar entries private.

Just as we’re dedicated to innovation when it comes to making better, more useful products, we’re also committed to finding new ways to educate you about how to control what information you share when using our products, and with whom. This series, along with the other videos on our YouTube Privacy Channel, are part of this awareness-raising effort. So watch the videos (including our very own blooper reel) and tell us what you think. And we'll be adding new videos to the Privacy Channel now and again, so be sure to check back.

A growing sense of doom washed over me as I read Steve's latest post

As a few people have noticed, this went out over RSS but was not on my site. The reason is that I “pulled” it. I started out thinking I would include some choice quotes from Steve’s essay and leave it at that, but… At some point I got enthusiastic and decided I wanted to say something about the culture of moving dirt (I think I have run out of things to say about Java the language, thank goodness). However, reality in the form of interesting work at work popped up, so I decided to put the essay off.

Bad luck, the post was already out on RSS. So here it is. I would like to rewrite it fully, just not right now. Sorry for letting it out onto RSS in such a half-baked form. I realize your time is valuable, and I think you deserve better from me.



I happen to hold a hard-won minority opinion about code bases. In particular I believe, quite staunchly I might add, that the worst thing that can happen to a code base is size…

People in the industry are very excited about various ideas that nominally help you deal with large code bases, such as IDEs that can manipulate code as “algebraic structures”, and search indexes, and so on. These people tend to view code bases much the way construction workers view dirt: they want great big machines that can move the dirt this way and that. There’s conservation of dirt at work: you can’t compress dirt, not much, so their solution set consists of various ways of shovelling the dirt around…

It’s just a mountain of dirt, and you just need big tools to move it around. The tools are exciting but the dirt is not…

The problem with Refactoring as applied to languages like Java, and this is really quite central to my thesis today, is that Refactoring makes the code base larger. I’d estimate that fewer than 5% of the standard refactorings supported by IDEs today make the code smaller. Refactoring is like cleaning your closet without being allowed to throw anything away. If you get a bigger closet, and put everything into nice labelled boxes, then your closet will unquestionably be more organized. But programmers tend to overlook the fact that spring cleaning works best when you’re willing to throw away stuff you don’t need

Design Patterns was a mid-1990s book that provided twenty-three fancy new boxes for organizing your closet, plus an extensibility mechanism for defining new types of boxes. It was really great for those of us who were trying to organize jam-packed closets with almost no boxes, bags, shelves or drawers. All we had to do was remodel our houses to make the closets four times bigger, and suddenly we could make them as clean as a Nordstrom merchandise rack.

A design pattern isn’t a feature. A Factory isn’t a feature, nor is a Delegate nor a Proxy nor a Bridge. They “enable” features in a very loose sense, by providing nice boxes to hold the features in. But boxes and bags and shelves take space. And design patterns – at least most of the patterns in the “Gang of Four” book – make code bases get bigger. Tragically, the only GoF pattern that can help code get smaller (Interpreter) is utterly ignored by programmers who otherwise have the names of Design Patterns tattooed on their various body parts

If you begin with the assumption that you need to shrink your code base, you will eventually be forced to conclude that you cannot continue to use Java. Conversely, if you begin with the assumption that you must use Java, then you will eventually be forced to conclude that you will have millions of lines of code…

You should take anything a “Java programmer” tells you with a hefty grain of salt, because an “X programmer”, for any value of X, is a weak player. You have to cross-train to be a decent athlete these days. Programmers need to be fluent in multiple languages with fundamentally different “character” before they can make truly informed design decisions

Java is like a variant of the game of Tetris in which none of the pieces can fill gaps created by the other pieces, so all you can do is pile them up endlessly.

Imagine that you have a tool that lets you manage huge Tetris screens that are hundreds of stories high. In this scenario, stacking the pieces isn’t a problem, so there’s no need to be able to eliminate pieces. This is the cultural problem: they don’t realize they’re not actually playing the right game anymore…

Java-style IDEs intrinsically create a circular problem. The circularity stems from the nature of programming languages: the “game piece” shapes are determined by the language’s static type system. Java’s game pieces don’t permit code elimination because Java’s static type system doesn’t have any compression facilities – no macros, no lambdas, no declarative data structures, no templates, nothing that would permit the removal of the copy-and-paste duplication patterns that Java programmers think of as “inevitable boilerplate”, but which are in fact easily factored out in dynamic languages.

Dynamic features make it more difficult for IDEs to work their static code-base-management magic. IDEs don’t work as well with dynamic code features, so IDEs are responsible for encouraging the use of languages that require… IDEs. Ouch…
—All quotes from Code's Worst Enemy by Steve Yegge

I have read many essays taking the minority position and explaining what is wrong with the “Java Programmer” mindset (as opposed to the “Programmer who happens to use Java mindset”), but Steve’s essay laid the cultural problems bare for me. He explains the technical issues well, but where this essay really shines is explaining the cultural issues and how they work with the technical issues to create a vicious cycle.

Of course I recommend reading the original. But may I add, please do not get sucked into arguing whether Design Patterns are good, or whether IDE refactorings really work, or any of the other technical points that are so much fun to rehash for the millionth time.

Instead, consider the cultural forces at work. Cultural problems cannot be solved with technology. If you are an advocate for change, ask yourself what sort of cultural change is needed, not what sort of technical problems need to be solved.

And oh yes… What do you think of his central thesis?

Year-End Giving

Whether it’s holidays or the close of the tax year that motivates, it’s a good time for generosity to non-profit causes. Here are some of mine:

The Tor Project, Inc.The Tor Project, Inc. develops anonymity software to help users navigate the web without being tracked — helping whistleblowers blog anonymously and dissidents browse past national firewalls. The Tor Project will be concluding its first year as a 501(c)(3) non-profit, and I’m proud to be on its Board of Directors. While your there, download the Tor software and add a node to the anonymity network.

 

Electronic Frontier FoundationElectronic Frontier Foundation relies on members’ donations to champion online freedoms in the courts. Its litigation against AT&T for participation in warrantless wiretapping has helped to expose the depth of the administration’s evasions. While you’re there, keep the pressure on congress to let the public hold telecommunications carriers accountable when they break the law.

 

Creative CommonsCreative Commons celebrates five years of helping creators to share their works and the public to find them, through standard copyright-permissions licenses. Science Commons is extending this spirit to enable web-like collaboration in scientific research. While you’re there, search for CC-licensed work or license a work of your own.

 

One Laptop Per ChildOne Laptop Per Child Through the end of the year, donors to OLPC can give one and get one — give one laptop to a child in a developing country and get one of their rugged yet open-source machines for your (inner) child.

 

Free Software FoundationFree Software Foundation produces both great software and the original great legal hack — copyleft. Grab some free software while you’re there.

 

ACLUAmerican Civil Liberties Union isn’t tax-deductible, but we need allies lobbying for our rights in Washington too.

 

 

Amnesty InternationalWGBHPlanned ParenthoodRhizomeOpen Rights GroupWitnessThe Harvard Crimson Financial Aid FundNature ConservancyWikipedia

I am (not) Spock


Tim Bray wants to know if/why spock.com matters. Here’s why I think it does. At some point, people are going to throw up their hands in disgust when invited to sign up for yet another service in order to assert or defend their online identities. So, for example, Spock thinks that Jon Udell is the inspector general at the Department of Justice, based on these two blog postings of mine. In fact, that person (whom I will not name here in order to avert yet more identity confusion1) is represented thusly in Spock.

I have no interest whatsoever in setting Spock straight about these facts, because I know that effort won’t carry over to ZoomInfo or to anywhere else.

I have a huge interest in establishing a presence, anchored somewhere in the emerging identity metasystem, to which I can refer Spock and ZoomInfo and other services. If Spock inspires other folks to appreciate why they might want to establish such presences for themselves, that’d be great. And based on some of the reactions I’m seeing, perhaps Spock will help us get there.

Isn’t it delightful, by the way, that both of these books exist?

I am Spock / I am not Spock


1 Of course, by writing the phrase “Jon Udell is the inspector general” I am probably ensuring that it will show up here.

Beth Noveck on WikiGov

Writes Beth in an excellent article in Democracy Journal:

Now, however, new technology may be changing the relationship between democracy and expertise, affording an opportunity to improve competence by making good information available for better governance.

She argues against relying on professionals to make good political decisions for us, and goes into some depth on the Patent Office’s Peer-to-Patent project, which she designed. (Thanks to Howard Rheingold for the link.)

[Tags: democracy beth_noveck wiki edemocracy everything_is_miscellaneous ]

On the shortest Python program


[First published in GOTOXO as Python-XO: On the shortest Python program on December 19, 2007.]

As reported in Caduceus Redux, Robert Bolgar says, “Maybe I should gear up my brain to learn some Python, too. About time I woke up to the computing world.”

I write this note to Josh and Zander Bolgar to provide a suggestion on how to help Dad wake up to the world of computing. Other members of the XO Generation may find it a way to help start their parent’s education on programming as well.

One of the most common sources of error in writing in a program is the failure to properly deal with what are known as “edge cases.” For example, running a loop once too often, or failing to run it far enough.

A further example is whether the program properly deals with the “null,” or empty case.

Since Python is based on set theory, let’s see if the Python compiler handles the case of null input properly. Linux provides many useful features, among them a perpetual source of nothingness in the form of a special device known as “/dev/null”. It appears as a file, but whenever you try to read it you get only the end-of-file signal. There is no data in that file save its end.

The question then is what happens when you execute:

$ python </dev/null

Yes! Python passes this test. Nothing in, nothing out. There is no error message, indicating that the Python compiler correctly recognizes null input as a valid program.

This program is also optimal in that no shorter program can be written, nor can any program take less time to compile and execute. Moreover, it is easy to construct a mathematical proof that this is so. For example,

Theorem: The empty program is the shortest possible program.
Proof: Any program that is not empty has at least one character, and is thus longer than the empty program.

Theorem: No program can be compiled faster than the empty program.
Proof: To read any program that is not empty, you must read at least one character and detect the end of input. The empty program requires only detecting the end of input, and so takes less time to compile than any other program.

“Maybe I should gear up my brain to learn some Python, too. About time I woke up to the computing world.”

I write this note to Josh and Zander Bolgar to provide a suggestion on how to help Dad wake up to the world of computing. Other members of the XO Generation may find it a way to help start their parent’s education on programming as well.

One of the most common sources of error in writing in a program is the failure to properly deal with what are known as “edge cases.” For example, running a loop once too often, or failing to run it far enough.

A further example is whether the program properly deals with the “null,” or empty case.

Since Python is based on set theory, let’s see if the Python compiler handles the case of null input properly. Linux provides many useful features, among them a perpetual source of nothingness in the form of a special device known as “/dev/null”. It appears as a file, but whenever you try to read it you get only the end-of-file signal. There is no data in that file save its end.

The question then is what happens when you execute:

$ python </dev/null

Yes! Python passes this test. Nothing in, nothing out. There is no error message, indicating that the Python compiler correctly recognizes null input as a valid program.

This program is also optimal in that no shorter program can be written, nor can any program take less time to compile and execute. Moreover, it is easy to construct a mathematical proof that this is so. For example,

Theorem: The empty program is the shortest possible program.
Proof: Any program that is not empty has at least one character, and is thus longer than the empty program.

Theorem: No program can be compiled faster than the empty program.
Proof: To read any program that is not empty, you must read at least one character and detect the end of input. The empty program requires only detecting the end of input, and so takes less time to compile than any other program.

Theorem: No program can be executed faster than the empty program.
Proof: The empy program is valid, yet requires no instructions be executed. It has no first instruction, it has no last instruction; it is just nothing.

“Maybe I should gear up my brain to learn some Python, too. About time I woke up to the computing world.”

I write this note to Josh and Zander Bolgar to provide a suggestion on how to help Dad wake up to the world of computing. Other members of the XO Generation may find it a way to help start their parent’s education on programming as well.

One of the most common sources of error in writing in a program is the failure to properly deal with what are known as “edge cases.” For example, running a loop once too often, or failing to run it far enough.

A further example is whether the program properly deals with the “null,” or empty case.

Since Python is based on set theory, let’s see if the Python compiler handles the case of null input properly. Linux provides many useful features, among them a perpetual source of nothingness in the form of a special device known as “/dev/null”. It appears as a file, but whenever you try to read it you get only the end-of-file signal. There is no data in that file save its end.

The question then is what happens when you execute:

$ python </dev/null

Yes! Python passes this test. Nothing in, nothing out. There is no error message, indicating that the Python compiler correctly recognizes null input as a valid program.

This program is also optimal in that no shorter program can be written, nor can any program take less time to compile and execute. Moreover, it is easy to construct a mathematical proof that this is so. For example,

Theorem: The empty program is the shortest possible program.
Proof: Any program that is not empty has at least one character, and is thus longer than the empty program.

Theorem: No program can be compiled faster than the empty program.
Proof: To read any program that is not empty, you must read at least one character and detect the end of input. The empty program requires only detecting the end of input, and so takes less time to compile than any other program.

Theorem: No program can be executed faster than the empty program.
Proof: The empty program is valid, yet requires no instructions be executed. It has no first instruction, no last instruction; it is just nothing.

I have written before about the shortest and longest sentences in natural language, in which a letter consisting of a single character (”!”) communicated a meaningful message. But Python is even better in that you can write a correct program by writing a program with no characters at all!

Zander and Josh can thus start their dad on his programming career by just having him execute the empty Python program.

If is of course much more fun to write programs that are longer and do some useful work, as I will try to show in future posts in this series. But education in any subject, including programming, must begin with a first step, even if it is just to show that “nothing ventured can still some education gain.”

ThoughtWorks is looking at Sweden

I am not sure how well it comes across in my blog posts, but joining ThoughtWorks have been the best move of my life. I can't really describe what a wonderful place this is to be (for me at least). I sometimes try - in person, after a few beers - but I always end up not being able to capture the real feeling of working for a company that is more than a company.

I'm happy at being with ThoughtWorks. It's as simple as that - I feel like I've found my home.

So imagine how happy I am to tell you that ThoughtWorks is exploring opportunities for an office in Sweden!

Now, I am one of the persons involved in this effort, and we have been talking about it for a while (actually, we started talking about it for real not long after I joined). But now it's reality. The first trips to Sweden will be in January. ThoughtWorks will be sponsoring JFokus (which is shaping up to be a really good conference, by the way. I'm happy to have been presenting there the first year). We will have a few representatives at JFokus, of course. I will be there, for example. =)

Of course, exploring Sweden is not the same thing as saying that an office will actually happen. But we think there are good reasons to at least consider it. I personally think it would be a perfect fit, but I am a bit biased about it.

So what are we doing for exploration? Well, of course we have started to look into business opportunities and possible clients. We are looking at partnerships and collaboration. We are looking at potential recruits. But really, the most important thing at this stage is to talk to people, get a feeling for the lay of the land, get to know interesting folks that can give us advice and so on. And that is what our travels in January will be about.

So. Do you feel you might fit any of the categories of people above? We'd love to meet you and talk - very informally. So get in touch.

This is exciting times for us!

Building the Spartan Wall

EAST LANSING, MI - Michigan State University officials today announced that they were placing the campus on “lockdown”, and never allowing the faculty, post-docs, graduate students, and select undergraduates to leave the campus again. In a statement released from the office of University President Lou Anna K. Simon, the University said that “We had recently taken the principled position that our faculty were under no circumstances allowed to use modern information sharing tools and collaborate online with their peers and the public. We felt that this position wasn’t strong enough. Our world class faculty and students carry important and critical University Intellectual Property and business records in their minds, and we just can’t let them get out and risk them sharing that property and exposing those records. We attempted pilot projects where each faculty member was given a stack of permission forms for each person they encountered to sign. That pilot was not successful. So we had to take the next logical step.”

“Not successful is an understatement”, said J. Allen Farnsworth, a tenured Classical Biological Anthropology professor, “I got a bloody black eye when I asked the lady at the lawn and garden shop to sign this form after I gave my lecture about the history of all art that includes references to caterpillar segments. The University told me that I had to do this any and each time I shared their Intellectual Property.”

Pressed for more information about such an unprecedented move, a spokesperson for Simon said that “we realize this may cause some consternation. But our IT department created this thing they call a ‘wallfire’ for our computers, and we really liked that wall idea. We absolutely must protect our Intellectual Property, and our Business Records. Especially all those emails from the facilities department about why my request to the move the light switch closer to my desk hasn’t yet been completed. However, we feel that our faculty and students will be well cared for and comfortable here. We have the finest in Spartan accommodations.”

“Spartan is right.”, agreed Professor Farnsworth, “all they gave us was a cheap plastic air mattress, a toothbrush, and some toilet paper. And it’s not even two-ply.”

Asked for additional comment, a spokesperson for David Gift, Vice Provost for Libraries, Computing, and Technology told us that “David couldn’t be more pleased. He’s taken a lot of criticism for being such a classicist on this issue. Forcing everyone to use our crappy custom developed webmail client really has been hard. He’s been having administrative meetings and lunches, and golf outings every day, to try to get the budget increased now that we told everyone they can’t use better products. And that’s hard, you know? If we don’t get more budget, we may never be able to back up our mail server ever again. By locking up all our faculty and students, which of course was the next logical step, it takes some of the pressure of me, er, him. And with Jack [Valenti, late head of the MPAA] gone, it’s been hard to find a sympathetic shoulder to lean on. I’m really doing everything I can for our fine University! I’m trying to protect our Intellectual Property! Oh, excuse me, I have to meet with some RIAA representative to install their monitoring software on our ResNet”

Asked why the Administration wasn’t covered by this policy, a spokesperson for Chris Hanna, Interim Assistant Vice President for Human Resources responded simply “well, we do keep some business records, but we’ve been assured that we are neither intellectual, or have any brains.”

A joint Engineering/Natural Sciences research initiative that would allow the faculty and students to dump the University contents from their brains each night showed some early promise, but some of the key neurological research needed to implement the filters properly, so that the faculty and students didn’t forget the names of their pets and partners was only available from Stanford. And with the new IT policies, MSU wasn’t able to share in that information for risk of exposing their own Intellectual Property.

So, for now, MSU faculty and students stay locked in their offices inventing new Pong games while waiting for the mail servers to be rebooted. Stay tuned for more on this developing story.

Thanks to Kevin Gamble for information contributing to this report

Holiday baking fun



Nothing evokes the spirit of the holidays more than the traditional foods that mark the season. Whether your holiday include latkes, a delicious butternut squash souffle, or even a deathless fruitcake, Googlers love to commemorate the occasion with food.

Last week I spent the afternoon baking Christmas cookies with a group of culinary-minded Googlers. Traditionally, this cookie is not just a festive holiday snack; it also makes a great homemade gift and an absorbing arts and crafts project. Our little get-together not only resulted in some melt-in-your mouth treats, but also allowed us to roll up our sleeves and get creative. Check out the photo album from our baking session, and try my recipe for buttery shortbread cookies.

Google Cookie Class

Buttery Shortbread Cookies

Preheat oven to 325 degrees.

Mix the following ingredients until smooth:
1-½ sticks unsalted butter (must be soft)
2-3/4 cups sugar
1 teaspoon Kosher salt

Then add the following ingredients and mix until smooth:
8 egg yolks
1/4 cup heavy cream
1 Tablespoon +1-1/2 teaspoon vanilla extract

Add:
8 cups all-purpose flour.

Mix everything thoroughly until dough comes together. Gather to a ball and flatten to a disc.

Chill the dough until firm. You might want to prepare the dough in the evening before you plan on baking and chill it overnight.

When chilled, roll to ¼-inch thickness, and cut into shapes with cookie cutters. Sprinkle tops with assorted sugars, jimmies and sprinkles.

Bake at 325 degrees F for 10-12 minutes on either a sheet greased with vegetable oil spray or a cookie sheet pan with parchment paper. Edges will be golden when they are done.

You may also bake the cookies plain, cool them, and then brush with icing sugar. Cover the whole surface of the cookie.

Serves: 4 dozen.

Banned from Technorati Top 100

OLPC arrives

My One Laptop Per Child computer arrived overnight, as if delivered by elves. As Drew Barrymore would say, “Magical!”

Other than saying that it’s the cutest thing since otters began holding hands, I feel incompetent to review it. I’ve only used it for a couple of hours, and it’s very different from the usual computers. I can say that it does some things incredibly easily, like take photos. On the other hand, I’m lacking the mental model for doing some more complex things, like downloading an ebook from the Web and reading it offline on the laptop.

But this was not a computer designed for me, so I’m going to give it time to shape my expectations. So far, though, it’s just so cute and cuddly that I want to feed it kibble and take it out for a walk four times a day.

[Anthropological note: The OLPC laptop serves the social function as puppies: When strangers see you with one, they just have to stop you, stroke it, and say “Awwww.”]

[Tags: olpc xo laptops linux ]

OGN5: Wednesday February 6, 2008

OGN5: Wednesday February 6, 2008. Great line-up for the next Oxford Geek Night: Rufus Pollock and Denise Wilton.

And Now, The Conclusion

Previously, on News You Can Bruise:

HORRID TRAGEDY IN PRIVATE LIFE, A CARTOON BY W.M. THACKERAY

We pick up the story with The History of "Punch", new to Project Gutenberg. They have a whole chapter on Thackeray, including the section "A Mysterious Picture", which describes the cartoon:

In 1847 (Volume XII., p. 59) Thackeray contributed a "social" picture which is to this day a wonder to all beholders. It is entitled "Horrid Tragedy in Private Life," and represents a room in which two ladies, or a lady and a servant, are in a state of the greatest alarm. [The maximum possible alarm was lower then -ed.] What the meaning of it all is there is nothing whatever to indicate (unless it be that something has fallen on the taller lady's dress); and on its appearance the "Man in the Moon" offered a reward of £500 and a free pardon to anyone who would publish an explanation. The reward was never claimed; and Thackeray's contribution remains one of Punch's Prize Puzzles, unsolved, and, apparently, unsolvable.

First, "Man in the Moon" is a great name for a magazine. Second, it's awesome that in those days magazines held the power of pardon. Truly it was a more, or possibly less, enlightened time.

Now, searching on the cartoon's name gets you to The Works of William Makepeace Thackeray with bibliographic commentary and nostalgia by his daughter, Anne Ritchie:

Is it too late to claim the £500? The room was my father's study, where two little girls were found by him dressed up in various tablecloths and curtains. One was enacting a queen, and was ordering the rival sovereign off to instant execution, when he came home unexepectedly, and drew them then and there.

Jake Berendes came closest to divining the purpose of the cartoon: 'i think the message is "there is a lot of nonsense you will have to put up with if you have children so don't bother".'

As The Hitherto Unidentified Contributions of W. M. Thackeray to "Punch", which I'd actually encountered on a previous cartoon hunt but given up on because I was looking for the wrong year, points out, "neither in drawing nor text is there any clue to the situation; nor, if there were, could the joke be considered a very funny one."

Indeed. This discovery opens up more questions, like, how did this cartoon get published?

Feed Maintenance

Lately, a couple of people have mentioned having trouble with my feeds. One person hadn't seen any new content from me since 2006 because he was subscribed to the old articles feed on my homepage. I 301 redirected it to this blog, since I don't self-publish articles on my site anymore. Another person mentioned that the recent "Fix That Tranny" post continuously appears as updated when it is in fact not. I switched the main feed for the site (the one that FeedBurner consumes) over to ATOM and I'm hoping that helps fix that problem. Please leave me a comment if you are having these or other problems.

Which reminds me of something I've been meaning to mention. The only valid public feed address for my blog is: http://feeds.feedburner.com/obie. If you are reading this via a jroller feed address, please take a moment to switch it over. (Unfortunately, it has to be done manually because JRoller doesn't give me any redirection options. At some point in the near future I'm going to switch off of JRoller and I don't want to lose you.)

Thanks for reading!

links for 2007-12-19


© Robert S. Sutor for Bob Sutor's Open Blog, 2007.
Posted under: News.
Permalink | No comments | Add to del.icio.us

A Green Box For Learning


[First published as A Green Box For Learning on December 19, 2007.]

When I speak about the XO these days, and I do so often, I frequently begin a sentence with the phrase, “This little green box.”

Though my own XO hasn’t yet arrived, I realize that I had my own Green Box For Learning over fifty years ago, as I have described in On Libraries: The Ernie Pyle Memorial Home/Library. Here are some pictures of it from a postcard I found via eBay:

200712 038
The Ernie Pyle Library in Albuquerque, New Mexico

200712 037
Front of postcard

200712 039
Back of postcard

One of the reasons I write this blog is to hopefully provide some education, and perhaps even some inspiration, so that some of the members of the XO Generation will make it their career to be librarians in their own country, providing Boxes for Learning both using the XO Laptop and in the form of buildings where all are welcome to come find a book and read it.

Let me hear your standards body talk

Jeremy Keith's "Year Zero" beautifully explains why the W3C needs our backs, not our bullets.

Second Life book: “Creating Your World”

I just received a copy of Creating Your World: The Official Guide to Advanced Content Creation for Second Life. Since I did the rather long series on Second Life earlier this year, I’ve been thinking about doing a book on construction and scripting techniques, though as with most such thoughts of mine I haven’t had the time to execute on it. Therefore, when I saw this book last week on Amazon I was curious to see what it would cover that might have.

On a very quick scan, it definitely is worth owning if you are into building in Second Life but don’t know all the ins and outs. There is a substantial amount of material available on the web about SL and at least two different wikis full of information. A common criticism I have seen about SL printed material is that the material is all otherwise available online. Well, yes, but not in one place and not designed to be read in a sequential way. Yes you can hunt for information is you know what you are looking for, but sometimes you miss the big picture or don’t discover entirely different ways of approaching the same problem.

This book covers the topics of prims, textures, scripting, landscaping, and use of external programs like Poser to do animation. It’s a dense book of almost 400 pages and offers a lot of practical tips. Moreover, it will help you learn what you don’t know. By this I mean that SL is a much more profound technical environment than most people give it credit for. This book covers a lot of that ground and has some good worked examples that pull together a lot of the concepts and techniques.

So is this the last book that is needed on SL building techniques? No, I don’t think so. I think there are different ways of approaching the material that will still add value, such as A Carpenter-Who-Knows-How-To-Code’s Guide to Construction in Second Life or An Architect’s Guide to Design and Construction in Second Life (I made those up). For its breadth of information, tips, and examples, I think this is well worth the purchase if you plan to get more serious about building objects and making them do things in SL.


© Robert S. Sutor for Bob Sutor's Open Blog, 2007.
Posted under: Virtual Worlds.
Permalink | No comments | Add to del.icio.us

SDK now available for download

Before anything else, I just wanted to say thanks to everyone who met up with us at BUG+NY last night! We had a great time, good conversations, and some pix are online now.  Now onto the stuff you really care about.  I will start with a little note from Ken, the head of software for Bug Labs:

Many moons ago Peter and I met in a crowded starbox on a dark and stormy evening.  He had some wood blocks and visions of a computer program that made creating user-designed consumer electronics easy. There were some pictures and various notes.  Well, I'm proud to announce that the software that resulted from those early conversations has been publicly released!

Called the Dragonfly SDK, it is an Eclipse-based software development kit for the BUG.  It's designed as an easy-to-use embedded development environment with a built-in web application sharing website.  While we have not released the BUG hardware yet, a virtual one is included to play around with.  Install directions are here  or if you're an Eclipse pro here is the update site URL.  Also, we've included a minimal getting started guide here.

(thanks Ken!)

We consider the SDK in an open beta now.  Accompanying it are also discussion forums, a wiki, and BUGnet is now live (click here to register).  If you are wondering what these things have in common, it's the beginnings of our online community.  We are building out numerous components to really enhance the community experience in the future, including a BUGmodule recommendation/voting system and more.  Most important to us is that every aspect of our communications become two-way.

So, please comment on the blog, post on the forums, contribute to the wiki, and build applications for BUGnet!  We want to hear from you, tell us what you think, what you want, and what else we can do to make the BUG experience as good as possible.

Fireside Chat at Web 2.0 Berlin

By Tim O'Reilly

I'd forgotten I'd done this video interview during the Web 2.0 Expo in Berlin last month until Paul Andrews sent a link to my brother, with the comment: "Tim is always so refreshing because he can be so candid. Did you see this great wide-ranging video? Nobody else is pointing this stuff out because it might cost them a VC or market point."

Intruders.tv talks to Tim O'Reilly about social media, OpenSocial, Microsoft and Google market dominance, Facebook, genomics, Bubble 2.0 and, of course, the iPhone.

Yes, it's true that I said in the interview that Web 2.0 is a stupid name, that the companies trying to exploit the social graph are going about it all wrong, and that we're entering a period of consolidation where the big players are going to start trying to capture more value than they create. I had a bad cold, so I was perhaps a little more dour about the prospects for the future than I normally am. But I also talked about the potential still ahead of us -- in applications built on the social graph (as soon as they give the users real control over their own data), in genomics, and in the re-engagement of computing with the physical world.

The interview really was a "fireside chat," in a fabulous supper club in Berlin, at a party hosted by Felix Petersen of Plazes, Tariq Krim of Netvibes, and Rodrigo Sepulveda of Vpod.tv.

More patent follies: the easiest design element becomes a toll gate

The Peer to Patent project (which I’ve reported on before) just pointed me to a particularly broad patent that could encumber user interfaces on the web and desktops for years to come.

The idea in this patent, submitted by Yahoo!, is a clever little idea: if someone is starting to drag an icon or mail message somewhere, why not bring the mountain to Mohammed, so to speak? If she is dragging a photo, for instance, the browser or operating system can guess that she wants to open it with PhotoShop or the Gimp, and present that choice right next to the icon for the photo.

Yes, an idea worth experimenting with, and one that (as the patent application amply emphasizes) could turn up soon in desktop window managers, in applications such as mail programs, in Ajax-enabled web pages, on mobile devices, and all sorts of places. And Yahoo! wants a cut for supposedly thinking it up.

Discussed by a lawyer in his blog and presented in full on the Peer to Patent site, this application seems to be a tough one to crack, despite its simplicity. Only two comments are posted, and no actual instances of prior art (although one comment tentatively suggests on instance). If no prior art is found in 35 days, the comment period closes on Peer to Patent and little stands in the way of granting the patent.

I have nothing against acknowledging the brilliance of the interface designers at Yahoo!. If GUI users overcome their initial surprise at having targets pop up at them, smart drag-and-drop may prove to be a convenience. Amusingly, the author of the patent application presents it as much more–a radical improvement to user interaction. But this has no impact on the granting of the patent. A novel idea can get a patent no matter how small an increment over current practice it represents.

What’s more disturbing to me is the cavalier way in which a Yahoo! designer has picked off the easiest layer of a complex design and tries to set up a toll gate to profit from it. (More likely, exercising the patent on such a small advance would just hold back its adoption, so I imagine Yahoo! is just trying to beef up its cross-licensing patent portfolio.) A consideration of the work that would go into implementing smart drag-and-drop quickly reveals that it would require serious research into user interaction and extensive changes to many parts of an application or desktop to deal with design issues such as:

  • How does the system choose the targets to present to the user for each type of icon or other user interface element?
  • How should the system subdivide its elements so that relevant targets are grouped with elements that can be dragged?
  • Once heuristics for these choices are developed, what data structures store these targets, how are targets linked to the objects being dragged, and how can an association be made quickly so as to present targets almost instantly to preserve a comfortable user experience?
  • What hooks does the desktop have to present to applications so they can create seamless experiences such as the hypothetic mail program suggested in the patent application? How do applications need to be redesigned in order to categorize user interface elements that are presented to the desktop for this service?
Needless to say, the patent application offers no help in any of these areas. When you read proposals, beware of “could be’s,” with which this application is larded. Yes, all of these things “could be” done. If smart drag-and-drop prove attractive, developers of operating systems, desktop, browsers, and applications will solve the problems. But they may have to license the right to do so from Yahoo!.

If you’re a web developer or other person with experience in graphical interfaces, maybe you can prevent this. Check out the claims and see what prior art you can find.

Bug Labs Use of Eclipse and OSGi


Bug Labs is a company I have been watching over the last couple of months.  Besides having a great name, they are building a platform for consumer electronics using OSGi for the runtime and Eclipse for the SDK.  They just announced their Eclipse based SDK, Dragonfly is now available.

It is great to see them use OSGi as their component runtime platform and another great example of the power of OSGi.    It looks like they are using the Concierge OSGi  runtime but I wonder how easy it would be to also use Eclipse Equinox?

btw, Ken Gilmer from Bug Labs are also doing a session at the OSGi DevCon, held in conjunction with EclipseCon.   It will be interesting to see what they have to present.

Android - Simulate key strokes

Here are the before and after screen shot(s). We can use IWindowManager’s injectKeyEvent to simulate a series of key strokes package org.apache.hello; import android.app.Activity; import android.os.Bundle; import android.os.DeadObjectException; import android.os.ServiceManager; import android.util.Log; import android.view.IWindowManager; import android.view.KeyEvent; import android.view.View; import android.widget.Button; public class HelloApp extends Activity { final IWindowManager windowManager = IWindowManager.Stub.asInterface(ServiceManager.getService("window")); /** * Called with the [...]

24 years, 364 days

Immediately after college, I started graduate school in Mathematics in 1980. After a couple of years I decided to take a leave of absence because 1) the woman to whom I was engaged was going to start graduate school several hundred miles away, and 2) I wanted to rethink what I was doing in my career. In thinking about where to work, I decided to apply to IBM because I had worked there as a summer student. Since I was a known quantity to them, I figured I could start at a higher position and a higher salary.

The day I joined IBM was December 20, 1982. Tomorrow is my 25th anniversary with the company.

In that time I worked on optimization software for the internal telecommunications network; moved to IBM Research to work on symbolic mathematical computation, user interfaces, a compiler, and web-based scientific publishing; spent four years in the middle of my Research work finishing my Ph.D., with IBM picking up the tab; moved to the Software Group to commercialize use of XML in vertical industries; worked on web services and SOA; took a turn in Websphere marketing for a couple of years; and am now in my current gig.

Along the way I did marry that woman, and am still married to her with our 25th anniversary coming next summer. We’ve had two kids and seven cats (though not all at the same time). We’ve moved six times, getting older houses further and further north.

That is, I’ve lived and have had a variety of careers, I just happen to have worked for the same company for a pretty long time. I have no idea when I’ll retire from working in this industry and it seems like a long time from now, maybe twenty more years. When I was younger I couldn’t understand why people would retire because I was in a very academic frame of mind. Now I get the point (grin), but I think of it as more of a transition to doing something else constructive, like teaching or writing full-time.

In any case, 25 years doesn’t seem as long as it used to.


© Robert S. Sutor for Bob Sutor's Open Blog, 2007.
Posted under: Family.
Permalink | 2 comments | Add to del.icio.us

Norway mandates open formats

According to this article (Norwegian only, sorry) on digi.no the Norwegian government has mandated the use of open document formats from January 1st, 2009. I’ll give a brief overview of what the article actually says.

There are three formats that have been mandated for all documentation between authorities and users/partners, namely:

  • HTML for all public information on the Web.
  • PDF for all documents where layout needs to be preserved.
  • ODF for all documents that the recipient is supposed to be able to edit

Goverment, state and regional agencies, authorities and services may also publish in other formats, but they must always publish in one of these formats. The decree is retroactive, and by 2014 all documents published prior to this decree must have been converted and made available in one of the three formats.

While the decree doesn’t mandate any format for internal documentation, I still have hopes that every interested party will standardize on the same formats for internal use as well, and it is also my hope that a real competitive market for information systems is created.

Either way, for me as a dedicated user of Linux, proponent of open formats and standards, this is delightful. I’ll end it with a quote from our minister for information technologies, Heidi Grande Røys, with some emphasis added from my side:

Everyone should have equal access to public documents. From 2009, every citizen will be able to choose which software they want to use to get access to public information. The goverment’s decision will also improve the terms of competition between software providers. In the future, we will not accept that govermental agencies lock the users of public information to closed formats.

YAY!

Vote for Java6 on Leopard!

As mentioned previously a lot of Java developers on OSX are upset at Apple's silence as to its intentions with respect to the release of Java 6. There used to be a developer preview available, which was pulled recently with no indication as to when a replacement would be available. People like me who upgraded in the hope of having the latest and greatest - which we have been very patiently waiting for over a year for - are very disappointed. It creates all kinds of annoyances, like not being able to run Java Tutorial examples. Some who are working on Java 6 projects cannot use their computer easily, without resorting to installation of a separate OS in a virtual machine, to do their job. We all like OSX: its a beautiful easy to use Unix that usually really helps us get our work done. I have been very happily using it since 2004.

The first solution of course is to have our voice heard. One way to do this is to file a bug with Apple. Please do this! The only problem I have with it is that as opposed to the Java bug database which is completely open, the Apple bug database is completely closed. So there's no real way of verifying how many people have posted a report. We must therefore complement that action with an equal Open action. Following the noble example given to us by Nova Spivack, when he asked for people to make their voice heard in support of the Burmese people and got some real results, let us do the same to help Apple make the right decision.
Anybody who would like to support this issue in the blogosphere, should help post a blog with the string

13949712720901ForOSX

The first part of the string is the decimal notation for 0xCAFEBABE [1], the magic cookie for JavaClass files (thanks David for the number and the pointer to Fredericiana's photo). Then post similar instructions on your blog or point people here. Let's see how far this gets us! [2]

We should then be able to use any search engine, Google is a good choice, to search for this string [3], and hopefully motivate the managers at Apple to invest more time on Java and be more open about their plans with the community.

Your vote may also be an energizer to those groups that are starting to port the OpenJDK to OSX (via the mac java community).

Notes

  1. Oops I just noticed a mistake here. 13949712720901 in dec = 0xCAFEBABE405 in Hex. Even better. So that's CAFEBABE + the HTTP 405 Response, which means "Method not available". :-)
  2. If you know a foreign language then please translate the instructions and explanations so that more people can understand what is going on. Always post a link to some instructions. Language is a Virus, but it is most virulent when it is understandable and hyperlinked, of course.
  3. A search on Google Web returns more results - more than AllTheWeb or AltaVista - but Google Blog Search contains less duplicates. The real number of votes is somewhere between those two numbers, as some people are voting on their open source web sites, which are not always feed enabled. Simon is keeping count.
  4. Karussell is keeping a list of related articles.

Update

Tuesday Nov 13: Landon Fuller has been able to get a very nice hello world GUI app running on OSX using the FreeBSD jdk1.6 port. It runs under X Windows only. Excellent work!

Nov 20th, 2007: Dave Dirbin publishes the first beta release of the open source java 6. This campaign has gathered 105 blog votes if we count the results from Google Blog Search, placing it easily among the top 10 bug reports at the Java Bug database. The Google web search returns 256 results, which will contain the blog search, many duplicate pages pointing to blogs + some extra votes people may have placed on the web. I guess that those extra votes may pop this bug report up to the top 5 position.

Wednesday Dec 19: Apple has put a developer preview of Java 6 up on Apple Developer Connection. It is nice to see things progress on that side. As a result of this conflict, Java development on OSX has become a lot richer, with an open source JDK starting to compete with the closed one from Apple. This can only be good for both, and for developer and customer confidence in the platform.

Validator.nu

Validator InputDocumentSchemasPresetNoneDocBook 5.0b9RELAX NGSVG 1.1XHTML BasicXHTML 1.0 FramesetXHTML 1.0 StrictXHTML 1.0 Strict with partial WCAG 1.0 checksXHTML 1.0 Strict, SVG 1.1 plus MathML 2.0XHTML 1.0 Strict, Ruby, SVG 1.1 plus MathML 2.0XHTML 1.0 TransitionalXHTML 1.0 Transitional with partial WCAG 1.0 checksHTML5 (experimental)XHTML5 (experimental)XHTML5, SVG 1.1 plus MathML 2.0 (experimental)XSLT 1.0ParserAutomatically from Content-TypeXML; don’t load external entitiesXML; load external entitiesHTML; flavor from doctypeHTML5HTML 4.01 StrictHTML 4.01 Transitional Be lax about HTTP Content-Type Show Source

Code's Worst Enemy

I'm a programmer, and I'm on vacation today. Guess what I'm doing? As much as I'd love to tell you I'm sipping Mai Tais in the Bahamas, what I'm actually doing on my vacation is programming. So it's a "vacation" only in the HR sense – I'm taking official time off work, to give myself some free time to get my computer game back online. It's a game I started writing about ten years ago, and

Hyperdata in Sao Paulo

In the past week I gave a couple of presentations of Hyperdata illustrating the concept with demos of the Tabulator and Beatnik, the Hyper Address Book I am just working on.

The first talk I gave at the University of Sao Paulo, which was called at the last minute by Professor Imre Simon, who had led the Yochai Benkler talk the week before. It was a nice turnout of over 20 people, and I spoke at a more theoretical level of the semantic web, how it related to Metcalf's law, as explained in more detailed in a recently published paper by Prof. James Hendler, and how an application like Beatnik could give a powerful social meaning to all of this. I also looked at some of the interesting problems related to trust and belief revision that come up in a simple application like Beatnik, which touched a chord with Renata Wassermann who has written extensively on that field of the Semantic Web.
Many thanks to Prof Simon, for allowing me to speak. For a view from the audience see Rafael Ferreira's blog (in English) and Professor Ewout's blog (in Portuguese).

Yesterday I gave a more Java oriented technical talk at GlobalCode, an evening learning center in Sao Paulo, with a J2EE project on dev.java.net. I touched on how one may be able to use OpenId and foaf to create a secure yet open social network.
About 25 people attended which must be a really good turnout for a period so close to Christmas, when everyone is looking forward to the surf board present from Santa Claus, getting into their swimming trunks and paddling off to catch the next big wave. Well the really big wave that everyone in the know will be preparing for is the hyperdata wave. And to catch it one needs to practice one's skills. And a good way to do this is to help out with a simple application like Beatnik.
Thanks to Vinicius and Yara Senger for organising this.

Today's news headlines

JavaPolis

Last week i spoke about JAX-RS and Jersey at JavaPolis. The slides are here. Initially i was a bit worried that there were too many slides but i managed to present in 50 minutes with 10 minutes spare that was fully taken up with questions.

Rather embarrassingly i drew a complete blank (and ignored my own general advice on REST that i was imparting!) on one question related to managing users if user information should not be stored as part of session state. Thankfully a JRuby guy in the audience, Ola Bini, came to my rescue and described what Rails does!

A user can have state and thus be a resource that is accessible using a user specific URI. HTTP messages should be stateless so the information declaring the user of each request is supplied by the authentication HTTP request headers. Sounds easy but as i understand it the current Java APIs tend to encourage the developer to use HTTP session state. The area of HTTP authentication and the Java APIs is one that I need to do more research on and then write a blog on the subject as penance!

 

I enjoyed Martin Odersky's talk on Scala. However, i suspect this is not the right way to present Scala to those who have never seen and/or tried it before. One of my biggest gripes about Scala was the lack of documentation, looks like that has changed with the new Programing in Scala book that is available as an early edition.

Josh Bloch changed his talk and instead presented on the 'Closures Controversy". He was very convincing and i could be persuaded by a sugary syntactical improvement for anonymous inner classes if only because such classes are real pain to write. But Neal provides some counter balance in this blog. Still i don't think the counter balance is enough. Before the conference i was of the opinion that we should leave the Java language alone, not bolt on stuff for which there is an "impedance mismatch", and instead focus resources into improving JVM support for language features, like closures, in Scala, Groozy, JRuby etc. If anything the conference reinforced my opinion.

The OSGi presentation was interesting. Seems like more application servers are adopting the OSGi model and it seems a genuinely useful framework for development, management and versioning of software components for both small and large scale software systems. After this presentation i had some interesting discussions on the topic of the Java language and OSGi. I may well be leaning to the opinion that the work on Java modules and super packages is not necessary and instead the JVM should have an OSGi implementation embedded in it.

Your Ruby tests are memory leaks

The title says it all. The only reason you haven't noticed, is that you probably aren't working on a large enough application, or have enough tests. But the fact is, Test::Unit leaks memory. Of course, that's to be expected if it is going to be able to report results. But that leak should be more or less linear.

That is not the case. So. To make this concrete, lets take a look at a test that exhibits the problem:
class LargeTest < Test::Unit::TestCase
def setup
@large1 = ["foobar"] * 1000
@large2 = ["fruxy"] * 1000
end

1000_000.times do |n|
define_method :"test_abc#{n}" do
assert true
end
end
end
This is obviously fabricated. The important details are these: the setup method will create two semi large objects and assign them to instance variables. This is a common pattern in many test suites - you want to have the same objects created, so you assign them to instance variables. In most cases there is some tear down associated, but I rarely see teardown that includes assigning nil to the instance variables. Now, this will run one million tests, with one million setup calls. Not only that - the way Test::Unit works, it will actually create one million LargeTest instances. Each of those instances will have those two instance variables defined. Now, if you take a look at your test suites, you probably have less than one million tests all over. You also probably don't have that large objects all over the place. But remember, it's the object graph that counts. If you have a small object that refers to something else, the whole referral chain will be stopped from garbage collection.

... Or God forbid - if you have a closure somewhere inside of that stuff. Closures are a good way to leak lots of memory, if they aren't collected. The way the structures work, they refer to many things all over the place. Leaking closures will kill your application.

What's the solution? Well, the good one would be for test unit to change it's implementation of TestCase.run to remove all instance variables after teardown. Lacking that, something like this will do it:
class Test::Unit::TestCase
NEEDED_INSTANCE_VARIABLES = %w(@loaded_fixtures @_assertion_wrapped @fixture_cache @test_passed @method_name)

def teardown_instance_variables
teardown_real
instance_variables.each do |name|
unless NEEDED_INSTANCE_VARIABLES.include?(name)
instance_variable_set name, nil
end
end
end

def teardown_real; end
alias teardown teardown_instance_variables

def self.method_added(name)
if name == :teardown && !@__inside
alias_method :teardown_real, :teardown
@__inside = true
alias_method :teardown, :teardown_instance_variables
@__inside = false
end
end
end
This code will make sure that all instance variables except for those that Test::Unit needs will be removed at teardown time. That means the instances will still be there, but no memory will be leaked for the things you're using. Much better, but at the end of the day, I feel that the approach Test::Unit uses is dangerous. At some point, this probably needs to be fixed for real.

On this day in History

8 years ago today, Heather and I stood in front of friends and family consenting to be joined as one for the rest of our lives.

Many things have changed since then: how we look, how we dress and where we live just to name a few. We are now three instead of two since the birth of our son Jakob. There is a lot more grey hair, and in my case a lot more weight. But with all these things that have changed, a couple of important things have stayed the same.

Heather is still the most beautiful woman in the world to me... and to you if you know what's good for you. We still lay in bed and laugh before retiring for the night. When I hold her I still feel complete in a way that transcends explanation.

Thank you for 8 years full of wonder, sorrow and joy. I can only imagine what the next 8 will bring.

MeToday: Christmas Trees

Tis the season for perfection and frustration!

Yes it is time once again, for the Christmas Tree to go up at the Davis house and with it will come pain, suffering and very likely wailing and gnashing of teeth. Give the video a watch.

Horton Hears a Who!

[image]

The new Horton Hears a Who movie looks awesome, no? Dr. Seuss definitely works better in computer animation than those wonky live-action Cat in the Hat and Grinch movies. (I hope they re-do those in animation as well).

(This was also a test of the file-upload page I just added... I used Totem to snag an image capture from the movie trailer, a plug-in for Nautilus to resize it, and then use the new form to upload it. Pretty easy!)

-Russ

CouchDB Performance

Android Task List / Selector

Credit for finding out how to do this goes to dims, who has been doing lots of cool Android hacking lately:

http://davanum.wordpress.com/2007/12/18/android-task-manager-primitive-prototype/

If you're interested in Android and what it can do, you should be following his online journal!

As a way to keep pushing and testing Hecl, I decided to see if I could copy what he'd done in Hecl. It has indeed produced a few improvements to the Java code. That done, making it work was short and sweet:

set layoutparams [linearlayoutparams -new {FILL_PARENT WRAP_CONTENT}
set layout [linearlayout -new $context]
$layout setorientation VERTICAL

# Create ourselves some commands to access Android internals.
java android.app.ActivityManagerNative activitymanagernative
java android.app.IActivityManager iactivitymanager
java {android.app.IActivityManager$TaskInfo} taskinfo
java android.content.ComponentName componentname

java java.util.List javalist
set am [activitymanagernative getdefault]
set tasks [$am gettasks 10 0 [null]]
set len [$tasks size]
set tasklist [list]
for {set i 0} {< $i $len} {incr $i} {
    set baseactivity [[$tasks get $i] -field baseactivity]
    lappend $tasklist "Task: [$baseactivity getpackagename]"
}

$layout addview [textview -new $context -layoutparams $layoutparams \
         -text "Currently running tasks"]

set lview [basiclist $context $tasklist -layoutparams $layoutparams]
$layout addview $lview
$lview requestfocus

set callback [callback -new [list [list SelectTask]]]
$lview setonitemclicklistener $callback

heclcmd setcontentview $layout

proc SelectTask {parent view position id} {
    [activitymanagernative getdefault] movetasktofront $position
}

It could have been squeezed further by avoiding some of the temporary variables, and perhaps also by implementing a mapping between java.util.List and Hecl lists.

I think one of the next things I want to do is work on some infrastructure so that people will be able to actually work on scripts on their phone. That's sort of a departure from the past (most phone keyboards would be sheer torture for inputting long scripts), but it's a cool demo in any case.

XML entities

There are several solutions to the XML entities problem. We just have to pick one:

  • Extend the set of default entities of XML. This seems the most pragmatic choice to me, but I’m sure there will be quite some people who don’t like it.
  • Continue the hack of known public identifiers that will enable a set of entities to work. This is what we’re doing now though as you can see not all browsers recognize all “important” public identifiers yet.
  • Deploy UTF-8+names. This solution seems sub optimal to me, but it’s worthy of consideration.

YAP6 Operator: Negated Operators

Another article of the series “Yet Another Perl 6 Operator”

The design of Perl 6 includes some unification mechanisms to bring some extra power to syntax and developers. One of such artefacts is the notion of meta-operators. With them, it is possible to construct augmented operators from existing ones.

The first of such meta-operators we’re going to look at is the quite simple '!' negation prefix.

When applied to infix operators which produce a Bool result, they build a boolean-wise negated version.

== (numeric equality) / !== (numeric inequality)
eq (string equality)  / !eq (string inequality)
~~ (smart match)      / !~~ (smart "mismatch")
<  (less than)        / !<  (not less than)

A few of these have traditional shortcuts: like '!=' as a synonym of '!==' and 'ne' which is the same as '!eq'. But from the point of view of homogeneity, the negated operators are more easily decipherable.

With negated relational operators, many expressions and algorithms will get a more natural code, without compromising their meaning and readability.

$x !eqv $y     versus    ! ( $x eqv $y )

The precedence of any negated operator is the same as the base operator. To avoid visual confusion with the '!!' operator, it is forbidden to modify any operator already beginning with '!'.

Note. This article is an expansion of section “Negated relational operators” of Synopsis 03 .


Next article is due tomorrow (Dec 20, 2007).



LINKS

HTML5 FPWD by Christmas?

HTML5 just might be published as (early) Christmas present thanks to the new W3C HTML WG staff contact, Michael(tm) Smith, sending the request from the airport just before takeoff.

(Ogg being out of the specification is not too nice. Hopefully will hear the outcome of the video workshop soonish.)

Testing New Admin Interface

In case you hadn't seen before, I'm actually using my own PHP app for my blog now. It's a very simple site based on my own MVC framework, and it's nice, but it was literally half done, as I never got around to making any sort of admin/editing interface and was using ScribeFire's client to do all that.

I finally decided that though I liked Scribefire and the general concept of doing all my posting via a back-end API (XMLRPC in this case), I really needed to get a simple edit/admin page up so I could tweak stuff farther back in time when I see problems. Also I was thinking about actually getting the tags stuff added (it's in the DB, just not in the published pages).

Anyways, that's what this post is about... testing the new pages. Didn't take too long really, not sure why I didn't do it before!

:-)

-Russ

Prison Break

Misapplying book terms, Pylons, and the 'end-user'

Misapplying book terms, Pylons, and the ’end-user’. Ben Bangert responds to Adam Gomaa’s claim that Pylons lacks “conceptual integrity”.

Stirring up the ESB honey pot

My article on ESB seems to have kicked off some discussion:
I've also had some private mail saying I misrepresented JBI - I'm hoping that person will comment more publicly too - they made some interesting points that deserve a wider hearing.

Ruby Performance Snapshot

What happened was, I got a note from Matt (who runs the ongoing server), saying my traffic had suddenly jumped (to like 230G/month) and wondering why. I had no idea, so I whipped up a little Ruby script to break down traffic by resource and host and time-of-day and so on. Then I wondered which version of Ruby would run it fastest?

ElapsedUserSystem
Ruby 1.8.659.658.80.8
Ruby 1.959.958.80.8
JRuby 1.1b62.563.41.3
JRuby trunk43.544.51.0

Details

This is on a 2GHz Intel MacBook running Leopard. The Ruby is the one that comes with the Mac, the 1.9 was built from svn on 12/17. JRuby 1.1b and the build from trunk were both run with -J-server. I tried to run Rubinius but it wouldn’t build from today’s tarball.

The data is a week’s worth of ongoing logfile ending on December 9th; 222,755,573 bytes in 1,029,073 lines.

The code is just a quick answer-me-this-question hack; I didn’t think about it, I just banged it out. Shows the effects of me having been a Perl user for a decade or more: log.rb, stats.rb.

The Value of Inefficiency

I read a wonderful post by danah boyd discussing how inexpensive actions in social networks add little or no value (think of all the Zombie invites you got on Facebook until you deactivated your account in dismay [*].) And that reminded me of a rule adopted by the members of a mailing list I started some years ago.

If you want to add someone to the list, you post to the list with your intention to do so. Then you wait a week for objections. After the week passes, you can nominate them.

Why the wait? Back in the late 1990’s, email was still new to many of the members of the community for which the list was created. They didn’t check email hourly, or even daily. So if requests to add someone to the list were processed immediately, we ran the risk that we could add someone who another member of the list may have had an issue with, and they would not know until well after the person was added.

In practice, I think there has been only one objection to adding someone to the list since the rule was implemented. But it’s kept because we decided that we want to respect each other and not spring disruptive surprises on one another.

The list thrives, the members discuss and debate, but we haven’t suffered mass defections and flame wars.

[*] Boy, I’m glad I gave up Facebook before all the beacon craziness.

links for 2007-12-19

Post Install Issues with MS07-069 (IE6 on XPSP2)

After downloading the Internet Explorer Cumulative Security Update for December 2007, some customers using IE6 on Windows XP Service Pack 2 have experienced an unexpected crash or hang upon launching Internet Explorer.

This might occur while navigating to a website hosting considerable media content (for example: http://msn.com) resulting in Internet Explorer displaying a dialog that states “Internet Explorer has experienced a problem and needs to close”. If you experience this issue, implement the applicable workaround provided in the following knowledge base article:

For your security, we strongly recommend that you deploy the Internet Explorer Cumulative Security Update for December 2007.

Terry McCoy
Program Manager
Internet Explorer Security

A Different "P2P"

An obvious test of whether something is a good idea is to look at who is supporting it. When the answer is a diverse set of competitors, you know it's an idea who's time has come. The Peer to Patent Project ("P2P") is a good example. Created by the NY School of Law in conjunction with the USPTO, this pilot initiative seeks to improve the quality of patents by opening up the patent review process to the public.

Currently, the USPTO is facing a backlog of nearly 800,000 pending patent applications. Given this number, patent examiners have limited time to undertake a thorough search of prior art to determine whether a claimed invention is truly "novel" . The result is an increase in the issuance of poor quality patents, which leads to an increase in litigation challenges and requests for re-examination.

Launched in June 2007, P2P attempts to tackle this problem through a community based review system. Participating inventors register and make applications publicly available on an open discussion forum. The public may then review each application and provide examples of prior art for consideration. These are then rated by relevance allowing the patent examiner to more efficiently focus on potential prior art that is most significant to an application.

Sun is an active supporter of P2P. The program's focus on patent quality (versus quantity) is consistent with how we make intellectual property investments. To this end, as part of P2P we have submitted applications in key areas such as "transactional memory" (an important component of Sun's chip multi-threading architecture), security and cryptography. In addition, we are encouraging our distinguished engineers and chief technologists to participate as peer reviewers for P2P. We also hope to provide funding support for P2P in the future.

Why have so many competing companies agreed to participate and test their innovation publicly? One reason is that the USPTO provides the benefit of an expedited review process to P2P participants. However, the more important explanation is that many companies believe that the current patent system has devolved from the intent of it's founders. And, it is to the benefit of all of us to see it improved.

What You've All Been Waiting For

Many of you have heard me boast that my cat, Fred, is able to sit on demand. Yet, very few of you actually believe me.

Ye of little faith. I am tired of getting harassed about this, so here's proof...

The story here is that I originally wanted a dog, but since my landlord won't allow them, I settled for a cat. And I'm glad I did, because I got a super smart one - one that I taught how to sit!

Below are 2 videos (no sound, unfortunately) of Fred sitting, per my request, for cat treats.

Download FredSitsForFood.AVI

Download FredSitsForFood2.AVI

Enjoy!/LC

Links for 2007-12-18 [del.icio.us]

Perl 5.10 released on Perl’s 20th anniversary

The Perl community is kind of insular; on weblogs, you’re much more likely to hear of Ruby and Python, despite the persistent popularity of Perl and its ever vibrant community. So I thought I’d lend my favourite language more of a voice and post about this latest big news.

Exactly 20 years after December 18, 1987, the day of Perl 1’s release, we get Perl 5.10. That’s not just a bugfix release – there’s tons of new stuff. Features I personally can’t wait to use:

  1. Looming tall above everything else in the list is the new Hash::Util::FieldHash core module. This finally allows you to write classes in Perl with proper encapsulation of instance variables and compile-time checking of typos in them. Perl 5 OO finally departs from the dark ages of global-in-disguise, implicitly-declared-at-first-use instance variables (catching up to where lexically scoped variables have been since Perl 5.0.0 in 1994).

    I don’t really know much about Python. I only stole its object system for Perl 5. I have since repented.Larry Wall

  2. Next on my list are the various pattern matching enhancements; in spite of its relative smallness, my favourite one is probably the \K escape, which I’ve long enjoyed using in Vim (where it is known as \zs) for how it can simplify a lot of substitutions. Next are named captures. Really, though, all of the new features are very cool.

  3. It seems silly to look forward to say, but Perl is meant to scale down just as well as up, and since say simplifies the simplest uses of print it’s going to make one-liners just that smidgen easier.

  4. The smart match operator, which makes many kinds of comparisons on compound data structures much more concise, and also serves as the basis for the new switch-like construct (one of the most requested features for Perl, ever), called given/when. Both were nicked from Perl 6.

  5. Oh yeah, the new “_” prototype obsoletes a module I wrote and put on CPAN. Deleting code is the greatest pleasure in the life of a programmer.

Oh, and if you’re on Windows (my condolences), keep an eye on the Strawberry Perl homepage, where a version of Perl 5.10 bundled with a C compiler and everything else required to install modules directly from CPAN will soon be available.

Links for Today: Mashup Evolution, BIG party, Web 2.0 and Process, and Hannah's New Trick

  • Andreas Krohn, one of Kapow's product managers, has a great blog posting about the evolution of mashups, from "Hacking to Assembling." He talks about several mashup products and gives Mashup Hub a plug here as well. We're going to have a lot more to talk about in 2008 around Mashup Hub and Info 2.0, so stay tuned... can't give away anything this time guys, sorry :)
  • For those of you who live in San Francisco, The Bay Area Interactive Group is co-sponsoring (with Yahoo) a community party that benefits "One Laptop per Child" this Thursday, Dec. 20th. They also plan to have a panel discussion around community and how to build a network of influencers to support your product. It looks interesting; I am hoping to attend and meet some folks. So if you're in town, come on out. It's $20/head, but for a good cause. Plus 200+ cool folks look to be attending.
  • Andrew Gent has an interesting blog post addressing the lack of process in Web 2.0. His argument is that Web 2.0 is about people, and that it involved technology, but there is a huge gap in the area of "process." I've been thinking a lot about this, and talking to my team about creating some sort of blueprint for success around how to implement/use/benefit from Web 2.0 in your company. We've already implemented a bunch of Web 2.0 products internally, and hopefully you'll be seeing this blueprint in Q1. We're talking to more customers though first, so if you have thoughts about what should be in here, let me know. Ideally I'd like to see this as a Web 2.0 Community effort - not just a vendor driven one.
  • Last, but certainly not least, here's a pic of my kitten Hannah from a few nights ago. She doesn't like it when I work too much and ignore her (I think she's co-dependent), so she snags my blackberry in an attempt to get attention. I snapped this pic right after she tired herself out and fell asleep on top of the device. Enjoy!              Hannah_bberry_2

Footnotes