Difference between revisions of "Hadoop HUJI: Measurement Hook-Points"
From Lawa
Direwolf007 (Talk | contribs) m (moved Hook Points to Hadoop HUJI: Measurement Hook-Points) |
Aviadpines (Talk | contribs) |
||
| Line 1: | Line 1: | ||
| − | 1.Mapper Input: | + | == Hadoop 0.21 Hook Points== |
| + | |||
| + | 1. Mapper Input: | ||
* Location: org.apache.hadoop.hdfs.BlockReader.readChunk() | * Location: org.apache.hadoop.hdfs.BlockReader.readChunk() | ||
| − | 2.Reducer Input: | + | 2. Reducer Input: |
* Location #1: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToMemory() | * Location #1: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToMemory() | ||
| Line 9: | Line 11: | ||
* Location #2: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToDisk() | * Location #2: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToDisk() | ||
| − | 3.HDFS Writes: | + | 3. HDFS Writes: |
* Location: org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock() | * Location: org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock() | ||
| + | |||
| + | == Hadoop 0.20.2 (CDH3) Hook Points == | ||
| + | |||
| + | 1. (TBD) | ||
| + | |||
| + | 2. Reducer Input: | ||
| + | |||
| + | * Location #1: mapred.org.apache.hadoop.mapred.ReduceTask.ReduceCopier.MapOutputCopier.shuffleInMemory() | ||
| + | |||
| + | * Location #2: mapred.org.apache.hadoop.mapred.ReduceTask.ReduceCopier.MapOutputCopier.shuffleToDisk() | ||
| + | |||
| + | 3. HDFS Writes: | ||
| + | |||
| + | * hdfs.org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock() | ||
Revision as of 16:48, 9 May 2011
Hadoop 0.21 Hook Points
1. Mapper Input:
- Location: org.apache.hadoop.hdfs.BlockReader.readChunk()
2. Reducer Input:
- Location #1: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToMemory()
- Location #2: org.apache.hadoop.mapreduce.task.reduce.Fetcher.shuffleToDisk()
3. HDFS Writes:
- Location: org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock()
Hadoop 0.20.2 (CDH3) Hook Points
1. (TBD)
2. Reducer Input:
- Location #1: mapred.org.apache.hadoop.mapred.ReduceTask.ReduceCopier.MapOutputCopier.shuffleInMemory()
- Location #2: mapred.org.apache.hadoop.mapred.ReduceTask.ReduceCopier.MapOutputCopier.shuffleToDisk()
3. HDFS Writes:
- hdfs.org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock()
