Methodology

  Traditionally, the following four questions have been asked for evaluating machines in a particular domain.

1. What are the applications that form the benchmark suite?
2. What machines will be measured?
3. What is the criterion for performance evaluation?
4. How will the data be collected?

We add another question to the above list.

1. What input sizes will be used to exercise the applications in the benchmark suite?

• Selection of Applications
• Input Selection
• Selection of Machines
• Performance Criterion
• Data Collection