Hadoop HUJI: Measurement Hook-Points

From Lawa
Revision as of 17:48, 9 May 2011 by Aviadpines (Talk | contribs)

Jump to: navigation, search

Hadoop 0.21 Hook Points

1. Mapper Input:

  • Location: org.apache.hadoop.hdfs.BlockReader.readChunk()

2. Reducer Input:

  • Location #1: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToMemory()
  • Location #2: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToDisk()

3. HDFS Writes:

  • Location: org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock()

Hadoop 0.20.2 (CDH3) Hook Points

1. (TBD)

2. Reducer Input:

  • Location #1: mapred.org.apache.hadoop.mapred.ReduceTask.ReduceCopier.MapOutputCopier.shuffleInMemory()
  • Location #2: mapred.org.apache.hadoop.mapred.ReduceTask.ReduceCopier.MapOutputCopier.shuffleToDisk()

3. HDFS Writes:

  • hdfs.org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock()
Retrieved from "https://www.cs.huji.ac.il:80/wikis/MediaWiki/lawa/index.php?title=Hadoop_HUJI:_Measurement_Hook-Points&oldid=41"