Average puzzle scores (Problems)

by Don Lester, Wednesday, March 14, 2007, 06:42 (6852 days ago) @ markakeen

I was pondering what we would get out of something like this, and I am not sure I quite get what the gain would be. An average doesn't mean much unless you know something about the population comprising the average. Being able to say the first set of April's Intermediate was solved by 73% and the first set of March's Intermediate was solved by 54% doesn't tell us anything. Not really anyway.

One might assume the problems in March were more difficult than the ones in April, but it might also mean that someone with more skill who did April's puzzles did not do the puzzles in March. I don't get the impression the number of active users per skill set is particularly high, and with a small sample set it is not possible to draw any statistical conclusions because the margin of error will be too high.

Since programmer time is likely set by a static budget, given the choice between something like this being developed vs new content, I would opt for new content.