Main Page

Welcome to the LAWA Wiki

This site is dedicated to the arcane art of Hadooping (very) large data.

What is LAWA?

Please refer to: https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/LawaWiki:Community_Portal

Getting started

• Configuring a Hadoop pseudo-cluster with one node (For development purposes).
• Configuring a Hadoop cluster.

Building Hadoop

• Building Hadoop 0.21.x from source.
• Building Cloudera's Hadoop Distribution (CDH3) from source

The Hadoop 0.21 Innards

• Hadoop Startup scripts de-mystified.

Hadoop HUJI Modifications

• Hadoop HUJI: Measurement hook-points.
• Hadoop HUJI: Scheduler hook-points

The Hadoop 0.21 Test Cluster

The test cluster for Hadoop 0.21 development is now live and its NameNode and JobTracker (master node) is located on hadoop23@cs. If you wish to access the web applications for the NameNode and JobTracker, please follow these links from inside the CS network:

NameNode statistics:

http://hadoop23:50070

JobTracker statistics:

http://hadoop23:50030

Processing of ARCs

• Package to parse and process arc files

General information about our cluster

• Information about the main cluster

Related work

Disk-Locality in Datacenter Computing Considered Irrelevant
G. Ananthanarayanan, A. Ghodsi, S. Shenker, I. Stoica]

Reining in the Outliers in MapReduce Clusters using Mantri
G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, E. Harris