I have finished voting for this test round and I have a few thoughts about this
which I like to bring up.

In the past the voting, as most people here are aware of, was based on three
different scores, Artistic, Technical and Interpretation of the Topic. The
score was given on a scale from 1-20.


discussion about changing this and I am not sure it is a good idea.

Firstly, the way I used to cast my vote was based on the three categories, and
rarely, if ever, would I give the same number in each category. I think this
gives a much more accurate result as each category is valuated independently
and also the numbers from 1-20 give room for a much tighter and accurate score.

Secondly, most of the people who participate are amateurs who are in the process
of learning. Receiving votes in 3 different categories is a great help to them
to see how to improve their work. Receiving only an overall score 0-10 does not
tell people much about how they did well and what needs to be improved. Only a
few people have time to write comments and those few comments are often sketchy


Thirdly, to keep the continuity of the IRTC, the scoring system should at least
be compatible to the past. I have no idea if the IRTC statistics page
(http://exether.free.fr/irtc/) will be continued, but I have no reason to think
it is not. By changing the scoring to only one overall, continuing this project
is impossible.

In the old system, there also was a little chance for a tie, as no two people
would ever receive the precise same number of votes.

So why was this changed and who made this decision? Or is the voting system
still under construction in this test round?

Post a reply to this message

"Hildur K." <nomail@nomail> wrote:
> I have finished voting for this test round and I have a few thoughts about this
> which I like to bring up.
>
> In the past the voting, as most people here are aware of, was based on three
> different scores, Artistic, Technical and Interpretation of the Topic. The
> score was given on a scale from 1-20.
>

> discussion about changing this and I am not sure it is a good idea.
>
> Firstly, the way I used to cast my vote was based on the three categories, and
> rarely, if ever, would I give the same number in each category. I think this
> gives a much more accurate result as each category is valuated independently
> and also the numbers from 1-20 give room for a much tighter and accurate score.
>
> Secondly, most of the people who participate are amateurs who are in the process
> of learning. Receiving votes in 3 different categories is a great help to them
> to see how to improve their work. Receiving only an overall score 0-10 does not
> tell people much about how they did well and what needs to be improved. Only a
> few people have time to write comments and those few comments are often sketchy

>
> Thirdly, to keep the continuity of the IRTC, the scoring system should at least
> be compatible to the past. I have no idea if the IRTC statistics page
> (http://exether.free.fr/irtc/) will be continued, but I have no reason to think
> it is not. By changing the scoring to only one overall, continuing this project
> is impossible.
>
> In the old system, there also was a little chance for a tie, as no two people
> would ever receive the precise same number of votes.
>
> So why was this changed and who made this decision? Or is the voting system
> still under construction in this test round?

I too would like to agree with the aforementioned.  Speaking from the amateur's
point of view, the individualized scoring helped a great deal more on my first
submissions than did the (at times nonexistent) comments.  In addition,
personally I am finding it rather difficult to vote on a pure basis from one to
ten, and find myself giving a score in the three categories and averaging them.

I don't mean to sound complaining, but I do think that the old three category
system worked better.  In addition, nomination for the merits seems a bit
flawed, as there are some people who (and I hope this won't hold true anymore)
would sometimes just give the Notable For: checkboxes in comments their check,
without truly understanding.  Naturally this did not effect the honorable
mentions, but with the new system, this problem seems to arise.

Just my two lines of code (or polygons technically as I work in lightwave)

Nimish

Post a reply to this message

Nimish wrote:
> "Hildur K." <nomail@nomail> wrote:
>> In the past the voting, as most people here are aware of, was based on three
>> different scores, Artistic, Technical and Interpretation of the Topic. The
>> score was given on a scale from 1-20.
> 
> I too would like to agree with the aforementioned.  Speaking from the amateur's
> point of view, the individualized scoring helped a great deal more on my first
> submissions than did the (at times nonexistent) comments.  In addition,
> personally I am finding it rather difficult to vote on a pure basis from one to
> ten, and find myself giving a score in the three categories and averaging them.
> 

I recall a long debate about this topic where many people said that they 
seldom gave different scores in the three categories and the range 1-20 
was too large.  I've forgotten where this debate took place and can't 
locate it right now.  As I recall, though, we seemed to agree on a 10 
point range and checks for technical, artistic and conceptual.

In any event, I'm happy to change it if we can get a reasonable consensus.

David Buck

Post a reply to this message

David Buck wrote:
> Nimish wrote:
> 
>> "Hildur K." <nomail@nomail> wrote:
>>
>>> In the past the voting, as most people here are aware of, was based 
>>> on three
>>> different scores, Artistic, Technical and Interpretation of the 
>>> Topic. The
>>> score was given on a scale from 1-20.
>>
>>
>> I too would like to agree with the aforementioned.  Speaking from the 
>> amateur's
>> point of view, the individualized scoring helped a great deal more on 
>> my first
>> submissions than did the (at times nonexistent) comments.  In addition,
>> personally I am finding it rather difficult to vote on a pure basis 
>> from one to
>> ten, and find myself giving a score in the three categories and 
>> averaging them.
>>
> 
> I recall a long debate about this topic where many people said that they 
> seldom gave different scores in the three categories and the range 1-20 
> was too large.  I've forgotten where this debate took place and can't 
> locate it right now.  As I recall, though, we seemed to agree on a 10 
> point range and checks for technical, artistic and conceptual.
> 
> In any event, I'm happy to change it if we can get a reasonable consensus.
> 
> David Buck

Just tried out the voting system.  Personally I like it VERY much.  Much 
simpler and with a greater emphasis on comments. There will no doubt be 
resistence because of the rougher granularity, ten stars instead of 
twenty.  I think this is because people want to make fine 
discriminations within a center range so they don't feel they are being 
too nasty or committing. Really they are still just using about five or 
ten grades but because it is cradled in a larger range they don't feel 
so bad. It is a mental discipline that you are really making a ranking 
and someone comes last.  So you might as well give that someone a single 
star.  But this may be too great an illusion for people to overcome.

One thing.  I seem to recall that with the old system, someone had 
written a program that, if you downloaded the thumbs and images, it 
helped you organize your voting  giving you different sorts to show how 
you ranked the images against each other.  I found that program VERY 
helpful.  Being able to get a page that displayed the images in order of 
how you have ranked them would be a very helpful feature if ever 
possible down the road.

-Jim

Post a reply to this message

David Buck wrote:

> I've forgotten where this debate took place and can't 
> locate it right now.

That's probably because the thread dates from March 2008:
 
http://news.povray.org/povray.general/thread/%3C47d2ce11%241%40news.povray.org%3E/?ttop=315132&toff=400

> As I recall, though, we seemed to agree on a 10 point range and checks for
technical, artistic and conceptual.
> In any event, I'm happy to change it if we can get a reasonable consensus.

The use of the words "agree" and "concensus" in relation to any
discussion on IRTC rules and voting system is precarious at best ;)

Post a reply to this message