Main Page

Welcome to the LAWA Wiki

This site is dedicated to the arcane art of Hadooping (very) large data.

What is LAWA?

Please refer to: http://www.lawa-project.eu/index.php/about/

Getting started

• Configuring a Hadoop pseudo-cluster with one node (For development purposes).
• Configuring a Hadoop cluster.

Building Hadoop

• Building Hadoop 0.21.x from source.
• Building Cloudera's Hadoop Distribution (CDH3) from source

The Hadoop 0.21 Innards

• Hadoop Startup scripts de-mystified.

Hadoop HUJI Modifications

• Hadoop Kelvin: A System for Fine Grained Measurement of Hadoop Network Traffic

The Hadoop 0.21 Test Cluster

The test cluster for Hadoop 0.21 development is now live and its NameNode and JobTracker (master node) is located on hadoop23@cs. If you wish to access the web applications for the NameNode and JobTracker, please follow these links from inside the CS network:

NameNode statistics:

http://hadoop23:50070

JobTracker statistics:

http://hadoop23:50030

Processing of ARCs

• Package to parse and process arc files

Munin graphs on Hadoop

• Installation and configuration of Munin on Hadoop cluster

Related work

Disk-Locality in Datacenter Computing Considered Irrelevant
G. Ananthanarayanan, A. Ghodsi, S. Shenker, I. Stoica]

Reining in the Outliers in MapReduce Clusters using Mantri
G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, E. Harris