Data Collection

The confidence level of conclusions drawn from a set of measured data depends on the size of the data set. The larger the sample, the higher is the confidence level. To estimate the mean performance of a system with an accuracy of and a confidence level of , the needed number of observations n is given by [15] as

where

    = normal variate of the desired confidence level

   		 = observed mean

s 		 = observed standard deviation

We take a few observations and find the required number of iterations from the calculated mean and standard deviation. All our results are reported with an accuracy of % at a confidence level of 95%.

To make the experiments fair the following guidelines were used.

1. To reduce the setup overhead, example sizes were chosen such that the run time was at least of the order of seconds.
2. For applications which involved a lot of terminal I/O operations we found a difference in running time if the output was directed to a file instead of the terminal. Hence we chose to direct all outputs to a file.