I am marking some 50-question, 4-choice, multiple choice tests. (About 
20 of them.)  The scores are noticeably low, 50%, 64%, 38%, 44%, 52%,...
I begin to worry that the marking key I am using may be mismatched with 
the version of the test I am marking.  After about nine papers marked, 
just as I am about to review the key against the actual questions,...I 
get a score of 84%.  I am immediately and intuitively certain the key is 
  the correct one and that this test group is just unusually low 
scoring.  Is there any science that discusses this sort of phenomena?

Post a reply to this message

"Jim Charter" <jrc### [at] msncom> wrote in message 
news:4b75c3e5$1@news.povray.org...
>I am marking some 50-question, 4-choice, multiple choice tests. (About 20 
>of them.)  The scores are noticeably low, 50%, 64%, 38%, 44%, 52%,...
> I begin to worry that the marking key I am using may be mismatched with 
> the version of the test I am marking.  After about nine papers marked, 
> just as I am about to review the key against the actual questions,...I get 
> a score of 84%.  I am immediately and intuitively certain the key is the 
> correct one and that this test group is just unusually low scoring.  Is 
> there any science that discusses this sort of phenomena?

I don't know, but I'm pretty sure that process is the foundation of all 
modern organized religion.

--
Jack

Post a reply to this message

Captain Jack wrote:
> "Jim Charter" <jrc### [at] msncom> wrote in message 
> news:4b75c3e5$1@news.povray.org...
>> I am marking some 50-question, 4-choice, multiple choice tests. (About 20 
>> of them.)  The scores are noticeably low, 50%, 64%, 38%, 44%, 52%,...
>> I begin to worry that the marking key I am using may be mismatched with 
>> the version of the test I am marking.  After about nine papers marked, 
>> just as I am about to review the key against the actual questions,...I get 
>> a score of 84%.  I am immediately and intuitively certain the key is the 
>> correct one and that this test group is just unusually low scoring.  Is 
>> there any science that discusses this sort of phenomena?
> 
> I don't know, but I'm pretty sure that process is the foundation of all 
> modern organized religion.
> 
> --
> Jack
> 
> 
LOL I understand you.  That certainty that the appearance of 
intelligence cannot possibly be a random result.

Post a reply to this message

> I am marking some 50-question, 4-choice, multiple choice tests. (About 
> 20 of them.)  The scores are noticeably low, 50%, 64%, 38%, 44%, 52%,...
> I begin to worry that the marking key I am using may be mismatched with 
> the version of the test I am marking.  After about nine papers marked, 
> just as I am about to review the key against the actual questions,...I 
> get a score of 84%.  I am immediately and intuitively certain the key is 
>  the correct one and that this test group is just unusually low 
> scoring.  Is there any science that discusses this sort of phenomena?

Maybe the one who got 84% was cheating using the same key as you had :-)

Post a reply to this message

scott wrote:
>> I am marking some 50-question, 4-choice, multiple choice tests. (About 
>> 20 of them.)  The scores are noticeably low, 50%, 64%, 38%, 44%, 52%,...
>> I begin to worry that the marking key I am using may be mismatched 
>> with the version of the test I am marking.  After about nine papers 
>> marked, just as I am about to review the key against the actual 
>> questions,...I get a score of 84%.  I am immediately and intuitively 
>> certain the key is  the correct one and that this test group is just 
>> unusually low scoring.  Is there any science that discusses this sort 
>> of phenomena?
> 
> Maybe the one who got 84% was cheating using the same key as you had :-)
> 
Good point.

I think another explanation is that the key was correct but some static 
set of answers were incorrect so that all the marks were depressed by a 
constant amount.

Post a reply to this message

And lo On Fri, 12 Feb 2010 21:11:07 -0000, Jim Charter <jrc### [at] msncom>  
did spake thusly:

> I am marking some 50-question, 4-choice, multiple choice tests. (About  
> 20 of them.)  The scores are noticeably low, 50%, 64%, 38%, 44%, 52%,...
> I begin to worry that the marking key I am using may be mismatched with  
> the version of the test I am marking.  After about nine papers marked,  
> just as I am about to review the key against the actual questions,...I  
> get a score of 84%.  I am immediately and intuitively certain the key is  
>   the correct one and that this test group is just unusually low  
> scoring.  Is there any science that discusses this sort of phenomena?

Okay think what a number of students/result graph would look like if there  
were no questions, just random ticks of the box. You'd expect a peak  
around the 25% mark tapering off on either side.

Now if there are questions and the people taking the exam are expected to  
know the answers you'd expect the peak to move to the right. Just like  
asking a mathematics class -

What is 1+1?
a)1
b)2
c)3
d)4

And similar easy questions.

Now imagine a misprint so the question reads - What is 1*1? What would you  
expect to happen to the graph with the same class? The peak would move to  
the left. This would also be the equivalent of having the wrong answer  
grid.

So taking the results you've given I'm just guessing you've got a  
low-scoring class; or the 'wrong' answer grid you're using is very similar  
to the 'correct' one.

-- 
Phil Cook

--
I once tried to be apathetic, but I just couldn't be bothered
http://flipc.blogspot.com

Post a reply to this message

Phil Cook v2 wrote:

Welcome back, Phil *V2*
Where have you been?


-- 

Best Regards,
	Stephen

Post a reply to this message

And lo On Thu, 04 Mar 2010 20:55:00 -0000, Stephen <mca### [at] aoldotcom>  
did spake thusly:

> Phil Cook v2 wrote:
>
> Welcome back, Phil *V2*
> Where have you been?

Here and there. Got into the habit of using Firefox to write my blog when  
I use Opera to track this group; pretty much raised my head one day and  
thought "Hey has it really been that long since I've checked in?"

And "oh look a probability question, yummy"

I've also just remembered the other problem, Opera's stopped tracking  
threads automatically.. Might take a look at Thunderbird see if that's any  
better.

-- 
Phil Cook

--
I once tried to be apathetic, but I just couldn't be bothered
http://flipc.blogspot.com

Post a reply to this message

Phil Cook v2 wrote:

> So taking the results you've given I'm just guessing you've got a 
> low-scoring class; or the 'wrong' answer grid you're using is very 
> similar to the 'correct' one.
> 

Yes, a small, fixed number of wrong answers on the marking key would 
give the same result I suspect also.  But it is interesting to me how 
intuitively we discount the possibility that the one high score is a 
random outcome, but rather the proof that the key is okay, or mostly 
okay, and the class is low scoring.

Post a reply to this message

And lo On Wed, 17 Mar 2010 02:33:08 -0000, Jim Charter <jrc### [at] msncom>  
did spake thusly:

> Phil Cook v2 wrote:
>
>> So taking the results you've given I'm just guessing you've got a  
>> low-scoring class; or the 'wrong' answer grid you're using is very  
>> similar to the 'correct' one.
>>
>
> Yes, a small, fixed number of wrong answers on the marking key would  
> give the same result I suspect also.  But it is interesting to me how  
> intuitively we discount the possibility that the one high score is a  
> random outcome, but rather the proof that the key is okay, or mostly  
> okay, and the class is low scoring.

Ah, but that's psychology rather than probability ;-) But yes, I suspect  
it's because we're used to clusters cropping up naturally. When an extreme  
appears we assume intention. If it had been a really low score would you  
have taken that as proof that the key was wrong?

-- 
Phil Cook

--
I once tried to be apathetic, but I just couldn't be bothered
http://flipc.blogspot.com

Post a reply to this message