Difference between revisions of "Main Page"

From Lawa
Jump to: navigation, search
(Getting started)
 
(25 intermediate revisions by 3 users not shown)
Line 4: Line 4:
  
 
== What is LAWA? ==
 
== What is LAWA? ==
Please refer to: http://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/LawaWiki:Community_Portal
+
Please refer to: http://www.lawa-project.eu/index.php/about/
  
 
== Getting started ==
 
== Getting started ==
 
* [http://hadoop.apache.org/common/docs/r0.21.0/single_node_setup.html Configuring a Hadoop pseudo-cluster with one node (For development purposes).]
 
* [http://hadoop.apache.org/common/docs/r0.21.0/single_node_setup.html Configuring a Hadoop pseudo-cluster with one node (For development purposes).]
 
* [http://hadoop.apache.org/common/docs/r0.21.0/cluster_setup.html Configuring a Hadoop cluster.]
 
* [http://hadoop.apache.org/common/docs/r0.21.0/cluster_setup.html Configuring a Hadoop cluster.]
 +
 +
== Building Hadoop ==
 
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Building_Hadoop Building Hadoop 0.21.x from source.]
 
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Building_Hadoop Building Hadoop 0.21.x from source.]
 +
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Building_Hadoop_CDH3 Building Cloudera's Hadoop Distribution (CDH3) from source]
 +
 +
== The Hadoop 0.21 Innards ==
 +
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Start_Scripts Hadoop Startup scripts de-mystified.]
 +
 +
== Hadoop HUJI Modifications ==
 +
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Hadoop_Kelvin Hadoop Kelvin: A System for Fine Grained Measurement of Hadoop Network Traffic]
  
 
== The Hadoop 0.21 Test Cluster ==
 
== The Hadoop 0.21 Test Cluster ==
Line 21: Line 30:
  
 
http://hadoop23:50030
 
http://hadoop23:50030
 +
 +
== Processing of ARCs ==
 +
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/ArcFiles: Package to parse and process arc files]
 +
 +
== Munin graphs on Hadoop ==
 +
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Munin_for_Hadoop Installation and configuration of Munin on Hadoop cluster]
 +
 +
== Related work ==
 +
[http://www.cs.berkeley.edu/~ganesha/disk-irrelevant_hotos2011.pdf Disk-Locality in Datacenter Computing Considered Irrelevant]<br> G. Ananthanarayanan, A. Ghodsi, S. Shenker, I. Stoica]
 +
 +
[http://www.cs.berkeley.edu/%7Eganesha/mantri_osdi.pdf Reining in the Outliers in MapReduce Clusters using Mantri]<br>
 +
G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, E. Harris

Latest revision as of 12:27, 19 July 2013

Welcome to the LAWA Wiki

This site is dedicated to the arcane art of Hadooping (very) large data.

What is LAWA?

Please refer to: http://www.lawa-project.eu/index.php/about/

Getting started

Building Hadoop

The Hadoop 0.21 Innards

Hadoop HUJI Modifications

The Hadoop 0.21 Test Cluster

The test cluster for Hadoop 0.21 development is now live and its NameNode and JobTracker (master node) is located on hadoop23@cs. If you wish to access the web applications for the NameNode and JobTracker, please follow these links from inside the CS network:

NameNode statistics:

http://hadoop23:50070

JobTracker statistics:

http://hadoop23:50030

Processing of ARCs

Munin graphs on Hadoop

Related work

Disk-Locality in Datacenter Computing Considered Irrelevant
G. Ananthanarayanan, A. Ghodsi, S. Shenker, I. Stoica]

Reining in the Outliers in MapReduce Clusters using Mantri
G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, E. Harris