1.8 Measuring, Reporting, and Summarizing Performance
- Wall-clock time, response time, elapsed time: Latency to complete a task including disk accesses, memory accesses, I/O, OS overhead and more
- CPU Time: Time the processor is computing
- The time a processor is computing is the time the CPU is used for all various programs running and not just one
- Therefore response time seen by the user is not CPU time where response time only accounts for one specific program
- Benchmarks
- Should evaluate true performance on real life usage rather than synthetic benchmarks
- Do not use kernels, toy programs, synthetic benchmarks
- Often benchmark suites are used so that things like different implementations will not take into account the things that one can optimize to gain performance
- Reporting Performance Results
- Guiding principle for performance measurements should be reproducibility
- Summarizing Performance Results
pokhym 