Russell Beattie: I’d even speculate that the recent interest in Erlang is driven by these same forces. The same people who swore they’d never use Java again if they could help it are looking for a good replacement for it on the server. The one spot where the JVM has no real peers is in long running server-side processes that need to execute hard-coded business logic as scalably as possible. Despite the fact that the JVM is a resource hungry beast that’s ponderously slow to start up and happy to eat as much memory as you can throw at it, it’s still the only real game in town for this type of application... for now. The reason people are looking at Erlang is not because its beautiful syntax, great documentation, or up-to-date libraries. Trust me. It’s because the Erlang VM can run for long periods of time, scaling linearly across cores or processors filling the same niche that Java does right now on the server.
First, great article. I encourage everybody to read the full thing. I also like Brian McCallister's follow up.
Next, to dispel a few myths. Slashdot is written in Perl, seems to handle the load, and also seems to stay up. While there are a number of BitTorrent implementations, the original and (to the best of my knowledge) the most pervasive version is written in Python. Yahoo is a mix, but a good portion of it is written in PHP, with critical functions written in C. Twitter is written in Ruby, had early scalability issues, but seems to be past them. These are all examples of massively scalable applications.
Finally, that syntax issue keeps coming up. I continue to look past it, but perhaps I shouldn’t. I mean PHP isn’t exactly known for elegant syntax, and it seems to be doing OK. But perhaps a Scala inspired syntax on top of Erlang’s byte codes would make the solution more palatable. Note that I said Scala inspired, and not a Scala port. Unlike the CLR which was designed to be multi-language, and unlike the JVM which is in the process of being repurposed to be multi-language also, Erlang’s VM is designed from the ground up assuming that objects typically are immutable and serializable.
Sam, I think you’re missing an angle here: The kinds of Java systems Russell is describing are basically cost centers. The BDUF approach only works when management expects the system to them remain essentially static and frozen, and the scalability ideal is 2x transactions for 2x $.
Whereas the systems you’re talking about put more of a premium on ease of modification.
the Erlang VM can run for long periods of time, scaling linearly across cores or processors filling the same niche that Java does right now on the server....
Santiago has it right. Erlang offers a new way of programming for a lot of people. It will take practice. I’m not against the idea of a non-erlang syntax, but for example scala has assignment. That allows more complex code — is that a good thing? I don’t think so. The constraints of erlang’s single assignment are a good thing to help build better systems. You want to maintain state? Think in terms of small processes sending messages.
I’m not against the idea of a non-erlang syntax, but for example scala has assignment. That allows more complex code — is that a good thing? I don’t think so.
I think what you are saying is what I meant by “Note that I said Scala inspired, and not a Scala port.” In particular, I tried to go out of my way to point out “immutable and serializable”.
As one who nearly effortlessly switches from Python to Ruby to C# to C, clearly whether a language is curly bracket based or whitespace based doesn’t affect me much; but it seems to be important to others. *shrug*
The biggest problem I have with Erlang is clearly an addressable one: the documentation of the libraries, and the lack of good samples that can be quickly found by Google/MSN-Live/Yahoo!/Ask searches. And many of the libraries appear to be abandoned at 0.n versions.
For example, I’m clearly looking at XMPP, but not just as a server, but also as a client. Jabberlang is at 0.2 and holding since 2005.
This doesn’t exactly inspire confidence in me. I’ll probably build up my own library, both so that it can handle Subjects and Bodies with characters that could be interpreted as markup, and as an educational exercise.
The examples you gave (with the exception of bittorrent) probably do not have long-running processes, which I think is one of the key criteria for what Russell is talking about.
Well, implementing assignment of local and non-local variables by compiling to a functional language is not so hard. At least local variables. In erlang, non-local closures I guess would have to use ets “hashes”.
Either you do this or the language is still pretty “weird”, e.g. no loops. I’d just advise people to get out of their comfort zone and use erlang for a while. There is a reason it’s the way it is.
Ted Leung quotes Sam Ruby, who bursts a few bubbles for the “native threads are the answer” crowd: Next, to dispel a few myths. Slashdot is written in Perl, seems to handle the load, and also seems to stay up. While there are a number of...
The C-based uTorrent totally out-competed the original Python BitTorrent to the point that BitTorrent Inc. bought uTorrent and renamed it to BitTorrent. The reasons for this may have nothing to do with C or Python, though.
The reason BT started in Python is because that was the language that the the codebase Bram started with was written in. When he hit networking i/o and event management problems with the hacked up asyncore module that was being used it was time to move to Twisted. This only got them so far (primarily because the iocp reactor is treated like a red-headed stepchild by the Twisted team) and a move to a c/c++ version that got the job done was inevitable. Buying uTorrent probably provided the most bang for the buck vs. starting from scratch.
There were various reactions to Sun’s announcement of moving from SUNW to JAVA. Frankly, I never thought there will be any from the technical world. But there are interesting ones, reflecting more on future of Java. Russell Beatie explains why...
Sam Ruby is writing about Russell Beattie writing about Java and Erlang. Russell thinks Java needs an overhaul. I think that Java has reached the point where technical, community, and business forces well exert pressure on the language to evolve in...
Hey Sam, thanks for the linkage and thoughts. Actually, Dave Brondsema was right above when he said your examples aren’t exactly what I was thinking of. In fact I’d go as far as to say that the long running processes in the sites you mentioned are actually Apache and lighttpd, rather than Perl/PHP/Python. And though it is common for Ruby to be served from a bunch of Mongrel servers (as opposed to threads within Apache like mod_perl/mod_php or FastCGI gateways to disposable external processes), the fact that Ruby’s architecture hasn’t been known to scale well shows what I’m saying.
I’d be quite confident delivering both static and dynamic data with a Java server like Tomcat or an Erlang based server like YAWS. Same for something like IM (i.e. OpenFire or eJabberd). That’s what those VMs were designed for. However, I wouldn’t be so confident if I were to run my entire server using just Python or just Perl, and I definitely won’t be hosting a pure PHP web server any time soon.
Ted Leung on the Air » Blog Archive » Scalability != concurrency “It is certainly possible to scale a program written in any language - that’s a given. Especially when scaling = throwing more hardware at it. But there’s got to be a better...
There were various reactions to Sun’s announcement of moving from SUNW to JAVA. Frankly, I never thought there will be any from the technical world. But there are interesting ones, reflecting more on future of Java. Russell Beatie explains why...
"Twitter is written in Ruby, had early scalability issues, but seems to be past them."-->I read somewhere they switched to ejabberd to solve the scalability issues: [link]
Sam Ruby : ... I see a future in lightweight threads without locks or semaphores. I think Newsqueak might be an instantiation of that future. (And no, this isn’t a smalltalk deriviative; Newsqueak has nothing to do with Squeak.)...