P16: a blog by Matt Kangas home archive
08 Sep 2008

"disco", a new map-reduce framework - in erlang+python

Woo! Another alternative to Hadoop crawls out of Nokia Labs:

Disco - discoproject.org

It's yet another Map-Reduce framework. This time the core is written in Erlang, and tasks are implemented in Python. Data transfer between nodes happens simply via HTTP.

There's no DFS implemented at all yet. It seems intended to run on Amazon EC2, where you'd just use S3 as your data store.

This feels like it might be a good fit for our needs, since we're a Python shop, and I'd rather hack on Erlang than Java. :)