Difference between revisions of "Main Page"
Aviadpines (Talk | contribs) |
|||
| Line 31: | Line 31: | ||
http://hadoop23:50030 | http://hadoop23:50030 | ||
| + | |||
| + | == Processing of ARCs == | ||
| + | The ArcTool package is now processing arc files. I uploaded the Jar with sources on /cs/phd/ouaknine/process_arc.jar | ||
| + | Job run example: bin/hadoop jar ~/process_arc.jar ArcProcessing -libjars=/cs/phd/ouaknine/PhD/lawa/arcTools/heritrix-1.14.4.jar,/cs/phd/ouaknine/PhD/lawa/arcTools/fastutil-6.1.0.jar keren1GB/ /user/ouaknine/output/sum4 | ||
Revision as of 03:24, 19 May 2011
Welcome to the LAWA Wiki
This site is dedicated to the arcane art of Hadooping (very) large data.
Contents
What is LAWA?
Please refer to: https://www.cs.huji.ac.il/wikis/MediaWiki/lawa/index.php/LawaWiki:Community_Portal
Getting started
- Configuring a Hadoop pseudo-cluster with one node (For development purposes).
- Configuring a Hadoop cluster.
Building Hadoop
The Hadoop 0.21 Innards
Hadoop HUJI Modifications
The Hadoop 0.21 Test Cluster
The test cluster for Hadoop 0.21 development is now live and its NameNode and JobTracker (master node) is located on hadoop23@cs. If you wish to access the web applications for the NameNode and JobTracker, please follow these links from inside the CS network:
NameNode statistics:
JobTracker statistics:
Processing of ARCs
The ArcTool package is now processing arc files. I uploaded the Jar with sources on /cs/phd/ouaknine/process_arc.jar Job run example: bin/hadoop jar ~/process_arc.jar ArcProcessing -libjars=/cs/phd/ouaknine/PhD/lawa/arcTools/heritrix-1.14.4.jar,/cs/phd/ouaknine/PhD/lawa/arcTools/fastutil-6.1.0.jar keren1GB/ /user/ouaknine/output/sum4
