Classes are held on Tuesday 16-18 in Shprintzak 213
| meeting | material | exercises | |
|---|---|---|---|
| Purim 8-) | |||
| 1 | 17/3/09 |
Introduction:
(slides)
The experimental method: measurements, hypothesis testing, and reproducibility. Use of experimentation in engineering, performace evaluation, and algorithmics. Tichy, Should computer scientists experiment more? Computer, 1998. Feitelson, Experimental computer science: the need for a cultural change. Manuscript, 2005. |
Ex1: Simple graphs |
| 2 | 24/3/09 |
Statistical graphics:
(slides)
Data visualization. Representing experimental results graphically. Jain chap. 10, 11. Michael Friendly's Gallery of Data Visualization. gnuplot reference, Ploticus reference. |
Ex2: Showing more complex data |
| 3 | 31/3/09 |
Measurement:
(slides)
Resolution, precision, and accuracy. Repeated measurements. Removing outliers. Confidence intervals. Jain chap 4, 5, 11 Lilja chap. 4, 6, 7 |
Ex3: Basic measurement |
| Pesah | |||
| 4 | 21/4/09 |
Designing and using microbenchmarks. lmbench.
(slides)
Using the average, maximum, minimum, and median. Jain chap 12, 14 Lilja chap. 8 Staelin and McVoy, mhz microbenchmark Usenix, 1998 |
Ex4: Measurement-based modeling |
| Yom Hazikaron | |||
| 5 | 5/5/09 |
Linear regression.
Applying linear regression to transformed data.
(slides)
Handling censored data. |
Ex5: Distribution of patience |
| 6 | 12/5/09 |
Experiment design:
(slides)
Factors and their interactions. Simple experiment design, full factorial design. ANOVA. Jain chap. 16-19; Law/Kelton chap. 12 |
Ex6: 22 experiment design |
| 7 | 19/5/09 |
Workloads:
(slides)
Workload analysis and characterization, and data cleaning. Feitelson and Tsafrir, Workload sanitation for performance evaluation. IEEE Symp. Performance Analysis of Systems and Software, 2006 |
Ex7: What's wrong with this data? |
| 8 | 26/5/09 |
The arrival process: burstiness and cycles.
(slides)
Self-similarity. Correlation and locality of sampling. Paxson and Floyd, failure of Poisson modeling IEEE/ACM Trans. Networking, 1995 |
Ex8: Burstiness vs. Poisson |
| 9 | 2/6/09 |
Measuring and Modeling the Internet:
(guest lecture by Prof. Scott Kirkpatrick,
in English)
(slides)
Sources of data, questions of coverage, bias, and stability. Visualization tools. What is required for performance estimation? Can one infer mechanisms of growth and management from topology? |
Ex9: Internet mapping tools |
| 10 | 9/6/09 |
Experiments with users:
(slides)
Observing users, usability testing, interviews. |
Ex10: Usability of menus |
| 11 | 16/6/09 |
Experimental algorithmics:
(slides)
Analysis vs. experimentation in studying algorithms |
Ex11: The complexity of sorting |
| 12 | 23/6/09 | case studies (slides) | Ex12: Comparing first-fit, best-fit, and next-fit |