Difference between revisions of "Main Page"
From Lawa
Direwolf007 (Talk | contribs) (→The Test Cluster) |
Direwolf007 (Talk | contribs) |
||
| (27 intermediate revisions by 3 users not shown) | |||
| Line 4: | Line 4: | ||
== What is LAWA? == | == What is LAWA? == | ||
| − | Please refer to: http://www. | + | Please refer to: http://www.lawa-project.eu/index.php/about/ |
== Getting started == | == Getting started == | ||
| − | * [http://hadoop.apache.org/common/docs/r0.21.0/single_node_setup.html | + | * [http://hadoop.apache.org/common/docs/r0.21.0/single_node_setup.html Configuring a Hadoop pseudo-cluster with one node (For development purposes).] |
| − | * [http://hadoop.apache.org/common/docs/r0.21.0/cluster_setup.html | + | * [http://hadoop.apache.org/common/docs/r0.21.0/cluster_setup.html Configuring a Hadoop cluster.] |
| + | |||
| + | == Building Hadoop == | ||
* [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Building_Hadoop Building Hadoop 0.21.x from source.] | * [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Building_Hadoop Building Hadoop 0.21.x from source.] | ||
| + | * [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Building_Hadoop_CDH3 Building Cloudera's Hadoop Distribution (CDH3) from source] | ||
| − | == The Test Cluster == | + | == The Hadoop 0.21 Innards == |
| + | * [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Start_Scripts Hadoop Startup scripts de-mystified.] | ||
| + | |||
| + | == Hadoop HUJI Modifications == | ||
| + | * [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Hadoop_Kelvin Hadoop Kelvin: A System for Fine Grained Measurement of Hadoop Network Traffic] | ||
| + | |||
| + | == The Hadoop 0.21 Test Cluster == | ||
The test cluster for Hadoop 0.21 development is now live and its NameNode and JobTracker (master node) is located on hadoop23@cs. If you wish to access the web applications for the NameNode and JobTracker, please follow these links from inside the CS network: | The test cluster for Hadoop 0.21 development is now live and its NameNode and JobTracker (master node) is located on hadoop23@cs. If you wish to access the web applications for the NameNode and JobTracker, please follow these links from inside the CS network: | ||
| Line 21: | Line 30: | ||
http://hadoop23:50030 | http://hadoop23:50030 | ||
| + | |||
| + | == Processing of ARCs == | ||
| + | * [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/ArcFiles: Package to parse and process arc files] | ||
| + | |||
| + | == Munin graphs on Hadoop == | ||
| + | * [https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/Munin_for_Hadoop Installation and configuration of Munin on Hadoop cluster] | ||
| + | |||
| + | == Related work == | ||
| + | [http://www.cs.berkeley.edu/~ganesha/disk-irrelevant_hotos2011.pdf Disk-Locality in Datacenter Computing Considered Irrelevant]<br> G. Ananthanarayanan, A. Ghodsi, S. Shenker, I. Stoica] | ||
| + | |||
| + | [http://www.cs.berkeley.edu/%7Eganesha/mantri_osdi.pdf Reining in the Outliers in MapReduce Clusters using Mantri]<br> | ||
| + | G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, E. Harris | ||
Latest revision as of 12:27, 19 July 2013
Welcome to the LAWA Wiki
This site is dedicated to the arcane art of Hadooping (very) large data.
Contents
What is LAWA?
Please refer to: http://www.lawa-project.eu/index.php/about/
Getting started
- Configuring a Hadoop pseudo-cluster with one node (For development purposes).
- Configuring a Hadoop cluster.
Building Hadoop
The Hadoop 0.21 Innards
Hadoop HUJI Modifications
The Hadoop 0.21 Test Cluster
The test cluster for Hadoop 0.21 development is now live and its NameNode and JobTracker (master node) is located on hadoop23@cs. If you wish to access the web applications for the NameNode and JobTracker, please follow these links from inside the CS network:
NameNode statistics:
JobTracker statistics:
Processing of ARCs
Munin graphs on Hadoop
Related work
Disk-Locality in Datacenter Computing Considered Irrelevant
G. Ananthanarayanan, A. Ghodsi, S. Shenker, I. Stoica]
Reining in the Outliers in MapReduce Clusters using Mantri
G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, E. Harris
