|
![](/i/fill.gif) |
It's an old problem. You can't measure something, so you try to estimate
it. But how do you figure out /how accurate/ your estimate is?
Computer graphics is full of situations where you want to estimate the
integral of something. The way you usually do this is to sample it at
lots of points and then take the weighted sum. The more points you
sample, the better the estimate. But usually each sample costs computer
power, so you don't want to take millions of samples except when it's
really necessary. But how do you know if it's "really necessary"?
It's a similar situation with benchmarking. You can run a benchmark and
time it. But what if Windows Update happened to run in the background
just at that moment? Or one of your cores overheated and changed clock
frequency? Hmm, better run the benchmark 3 times and take the average.
Still, 3 flukes are three times less likely than 1, but still hardly
what you'd call "impossible". People play the lottery with worse odds
than that!
So many you run the benchmark 100 times. Now if all 100 results are
almost identical, you can be pretty sure your result is very, very
accurate. And if all 100 results are all over the place, you should
probably do a bazillion more runs and plot a histogram. Still, how do
you put a number on "how accurate" your results are?
Does anybody here know enough about statistics to come up with answers?
Post a reply to this message
|
![](/i/fill.gif) |