> The 700k population of students perhaps doesn't fluctuate much, but the
> difficulty of the questions surely does. But how to control for that?

Isn't that a perfect reason to have a fixed % of people getting each grade?

Post a reply to this message

On Tue, 27 Aug 2013 10:03:26 +0100, scott wrote:

>> The 700k population of students perhaps doesn't fluctuate much, but the
>> difficulty of the questions surely does. But how to control for that?
> 
> Isn't that a perfect reason to have a fixed % of people getting each
> grade?

Depends on whether you need an absolute measurement of skill/knowledge or 
a relative measurement.

Jim

Post a reply to this message

>>> The 700k population of students perhaps doesn't fluctuate much, but the
>>> difficulty of the questions surely does. But how to control for that?
>>
>> Isn't that a perfect reason to have a fixed % of people getting each
>> grade?
>
> Depends on whether you need an absolute measurement of skill/knowledge or
> a relative measurement.

If the distribution in skill/knowledge of the students is fluctuating 
less than the difficulty of the questions, it will give you a more 
accurate absolute measure (if the alternative is to give everyone grades 
based on the number of questions they get right).

Given there are only a handful of questions in an exam, and hundreds of 
thousands of students taking them (and hundreds of thousands of 
teachers), I'd say it's way more likely the distribution of question 
difficulty varies from year to year than that of the students' skill.

Anyway I'd suggest that the main use of school exam grades is to secure 
a place at a college or university, so a relative measure is probably 
all that's needed.

Post a reply to this message

On Wed, 28 Aug 2013 08:31:49 +0100, scott wrote:

>>>> The 700k population of students perhaps doesn't fluctuate much, but
>>>> the difficulty of the questions surely does. But how to control for
>>>> that?
>>>
>>> Isn't that a perfect reason to have a fixed % of people getting each
>>> grade?
>>
>> Depends on whether you need an absolute measurement of skill/knowledge
>> or a relative measurement.
> 
> If the distribution in skill/knowledge of the students is fluctuating
> less than the difficulty of the questions, it will give you a more
> accurate absolute measure (if the alternative is to give everyone grades
> based on the number of questions they get right).
> 
> Given there are only a handful of questions in an exam, and hundreds of
> thousands of students taking them (and hundreds of thousands of
> teachers), I'd say it's way more likely the distribution of question
> difficulty varies from year to year than that of the students' skill.
> 
> Anyway I'd suggest that the main use of school exam grades is to secure
> a place at a college or university, so a relative measure is probably
> all that's needed.

Well, maybe.  If you buy into the idea that pre-uni instruction degrading 
isn't a problem.  An absolute measure can be used to uphold an absolute 
measure, and if the pre-uni schools aren't meeting it, then they need to 
up their game, rather than the universities lowering their standards.

Jim

Post a reply to this message

>> Anyway I'd suggest that the main use of school exam grades is to secure
>> a place at a college or university, so a relative measure is probably
>> all that's needed.
>
> Well, maybe.  If you buy into the idea that pre-uni instruction degrading
> isn't a problem.  An absolute measure can be used to uphold an absolute
> measure, and if the pre-uni schools aren't meeting it, then they need to
> up their game, rather than the universities lowering their standards.

Just by the nature of setting questions I think the standard over a 
longer period (eg 10 years) would be easier to keep consistent. So you 
could still look at the absolute scores averaged over the long term to 
analyse the performance. Giving students relative scores will avoid them 
being penalised by the year-to-year variations in the difficulty of the 
questions.

Post a reply to this message