POV-Ray : Newsgroups : irtc.animations : comments on voting : Games Server Time
24 Nov 2024 08:49:29 EST (-0500)
  comments on voting : Games (Message 11 to 16 of 16)  
<<< Previous 10 Messages Goto Initial 10 Messages
From: Dan Connelly
Subject: Re: comments on voting : Games
Date: 27 Dec 1998 20:34:01
Message: <3686E049.46A712F2@flash.net>
Greg M. Johnson wrote:

> In my voting, I've also found myself thinking, "I don't care how good
> this one is; I don't think it should win," or, "I don't want to lose to
> THIS entry." Is this contrary to the spirit of the IRTC?  For example, I
> thought I'd much rather lose to 'Pool Shark' than 'Canyon', and so gave
> 'Pool Shark' near 20-20-20's. I suppose this philosophy could be taken
> to an unethical extreme, in giving everyone else's entry a 1-1-1. I'm
> certainly not doing that.

I find this very discouraging, as by so doing you are unfairly weighting your
vote.  I will struggle on choosing a correct distribution of points,
but by tossing around 20's, you thus make my votes, in which contestants
differ by a few points, of trivial relative significance.

Votes need to justify themselves.  If A scores higher than B in category
C, it is because A is better than B in C, not because one had
a generally good feeling for A overall.  Was Pool Shark an all-time
ground-breaking masterpiece in all three areas?  If so, the 20-20-20 is
justified.  But please don't "stuff the ballot box" to assert your own
general preference, indepent of the point categorization system.

Dan


Post a reply to this message

From: John VanSickle
Subject: Re: comments on voting : Games
Date: 1 Jan 1999 12:27:32
Message: <368D0602.595B3F26@erols.com>
Dan Connelly wrote:
> 
> Greg M. Johnson wrote:
> 
> > In my voting, I've also found myself thinking, "I don't care how good
> > this one is; I don't think it should win," or, "I don't want to lose to
> > THIS entry." Is this contrary to the spirit of the IRTC?  For example, I
> > thought I'd much rather lose to 'Pool Shark' than 'Canyon', and so gave
> > 'Pool Shark' near 20-20-20's. I suppose this philosophy could be taken
> > to an unethical extreme, in giving everyone else's entry a 1-1-1. I'm
> > certainly not doing that.
> 
> I find this very discouraging, as by so doing you are unfairly weighting your
> vote.  I will struggle on choosing a correct distribution of points,
> but by tossing around 20's, you thus make my votes, in which contestants
> differ by a few points, of trivial relative significance.
> 
> Votes need to justify themselves.  If A scores higher than B in
> category C, it is because A is better than B in C, not because one
> had a generally good feeling for A overall.  Was Pool Shark an all-
> time ground-breaking masterpiece in all three areas?  If so, the 20-
> 20-20 is justified.  But please don't "stuff the ballot box" to
> assert your own general preference, indepent of the point
> categorization system.

What I did on the last round was to rank the animations in each
category, which involves watching each of them a lot, one after the
other, to decided which of the ones was more technically precise,
artistic, better-conceived, etc, than the others.  There were 13 of
them in the last round (I think), so I ranked each of them from one to
13.  I then awarded points based directly on the ranking (last place
got four points, first place got 16).

The IRTC administrators might want to impose a kind of ranking scheme
in each category, so that someone won't give straight ones to every
animation but his own.

Your mileage may vary.

Regards,
John


Post a reply to this message

From: Greg M  Johnson
Subject: Re: comments on voting : Games
Date: 1 Jan 1999 13:28:00
Message: <368D1407.5BD06186@aol.com>
Dan Connelly wrote:

> I find this very discouraging, as by so doing you are unfairly weighting your
> vote.  I will struggle on choosing a correct distribution of points,
> but by tossing around 20's, you thus make my votes, in which contestants
> differ by a few points, of trivial relative significance.
>

There's probably good counsel in your post, Dan.

In the Stills, I usually give 20-20-20's to my favorite, and probably >16-16-16
to six or eight of my other favorites. And I do differentiate, especially among
the top 12 or so, between excellent scores in the different categories: one might
be 20-15-14 and another 14-15-20.  In that Animations round,  I think I gave Pool
Shark close to 20-20-20,  but I don't feel that I "stuffed the ballot box."  I'm
sure I gave >16-16-16 to two or three others, notably Rusty's Late.I've never
done the math, but I suspect that I give out a uniform distribution of rankings,
from near 0-0-0 to near 20-20-20.  I'm wondering if from your post, you typically
give out a normal distribution centered around 10-10-10, with just one or two
20-20-20's  per year.

I quote Dan again:

> Votes need to justify themselves.  If A scores higher than B in category
> C, it is because A is better than B in C, not because one had
> a generally good feeling for A overall.  Was Pool Shark an all-time
> ground-breaking masterpiece in all three areas?

Well, for example, there was once an entry that looked just like a photograph of
a group of people in bearksin working on building a temple.  The problem is that
these humans were all bald, pale, hairless, and identical septuplets.  That
problem  wrecked the image for me, both in its artistic appeal,  its attention to
technical detail, and its theme. I did not rate it highly in any category.  I
felt guilty about my vote, as I was sure some voters would be swept away by its
"looking just like a photograph."

Perhaps one solution to "rogue voters" is for the IRTC staff to put more
"experts" in the voting list for Animations, especially if we're only getting
10-15 entries.


Post a reply to this message


Attachments:
Download 'us-ascii' (3 KB)

From: Nieminen Mika
Subject: Re: comments on voting : Games
Date: 1 Jan 1999 14:48:45
Message: <368d269d.0@news.povray.org>
Greg M. Johnson <gre### [at] aolcom> wrote:
: In the Stills, I usually give 20-20-20's to my favorite,

  This is ok if the image really is excellent in all the three categories.
  I'm not saying that you are one of those, but it seems that there is people
out there who doesn't distinguish very much between the different categories.
If the image is astonishingly great, they give high points in _each_ category,
no matter how bad it may be in one or two of them. An image may be technically
almost perfect, photorealistic, incredibly perfect, but that doesn't mean
that it's artistically good or original (specially the last one, since most
of the great images I have seen have nothing original).
  Although an image looks great, it's not wrong to give it something like
20-18-2. If it has a total lack of originality, then you have to judge it
from that point of view and give points keeping that in mind. Technically
excellent doesn't mean it's artistically excellent or excellently original.
  And I'm sure you will never, ever see something like 1-2-20. If it's
technically awful, then almost everybody will give it very little points in
each category, no matter if the idea of the image is something nobody have
ever seen in this universe.

-- 
main(i){char*_="BdsyFBThhHFBThhHFRz]NFTITQF|DJIFHQhhF";while(i=
*_++)for(;i>1;printf("%s",i-70?i&1?"[]":" ":(i=0,"\n")),i/=2);} /*- Warp. -*/


Post a reply to this message

From: Dan Connelly
Subject: Re: comments on voting : Games
Date: 3 Jan 1999 22:44:37
Message: <36903920.5D62BD6D@flash.net>
"Greg M. Johnson" wrote:

> There's probably good counsel in your post, Dan.
> 
> In the Stills, I usually give 20-20-20's to my favorite, and probably >16-16-16 to
six or eight of my other favorites. And I
> do differentiate, especially among the top 12 or so, between excellent scores in the
different categories: one might be
> 20-15-14 and another 14-15-20.  In that Animations round,  I think I gave Pool Shark
close to 20-20-20,  but I don't feel that
> I "stuffed the ballot box."  I'm sure I gave >16-16-16 to two or three others,
notably Rusty's Late.I've never done the math,
> but I suspect that I give out a uniform distribution of rankings, from near 0-0-0 to
near 20-20-20.  I'm wondering if from
> your post, you typically give out a normal distribution centered around 10-10-10,
with just one or two 20-20-20's  per year.

Well, to be honest, my post was way too harsh.  To be fair, the rules
are intentionally vague on voting policy.  20-20-20's are your perogative.

I think all that can be asked is that the categories are honored -- that
a 20 "concept" not be given because an image has good technical content.

I've never given a 20-20-20.  I've given only a single 20 technical vote
ever ( Howard Day's "Canyon" animation, which I thought was in all ways
excellent technically, but fell somewhat shorter in concept, and thus 
wasn't the top ranked overall animation that round.  It also wasn't
20 artistically. ).  I don't recall ever voting 20 in concept or art.
I've given out a few 19's each round.

My votes work out to an 11-12 mean, with sigma near 3.... this makes
20's and 1's quite rare (a fraction of a percent).

 
> I felt guilty about my vote, as I was sure some voters would be swept away by its
"looking just like
> a photograph."

A photograph of clones, perhaps :).

I don't think you need to feel guilty on this one.

 
> Perhaps one solution to "rogue voters" is for the IRTC staff to put more "experts"
in the voting list for Animations,
> especially if we're only getting 10-15 entries.

If expertise was required for animation voting, I'd be first on the chopping block :).

Dan

-- 
http://www.flash.net/~djconnel/


Post a reply to this message

From: Dan Connelly
Subject: Re: comments on voting : Games
Date: 3 Jan 1999 22:49:23
Message: <36903A3B.4F2F3FB9@flash.net>
Nieminen Mika wrote:


>   Although an image looks great, it's not wrong to give it something like
> 20-18-2. If it has a total lack of originality, then you have to judge it
> from that point of view and give points keeping that in mind. Technically
> excellent doesn't mean it's artistically excellent or excellently original.
>   And I'm sure you will never, ever see something like 1-2-20. If it's
> technically awful, then almost everybody will give it very little points in
> each category, no matter if the idea of the image is something nobody have
> ever seen in this universe.

I generally agree with this, but note there is more than "originality"
to the third category than "originality".  From the FAQ :

[3.10] What do the categories mean? 
      Hard question. They mean whatever you want them to mean, in general. However,
our intent was to separate the purely
artistic and purely
      technical aspects of these unique images so they can be rated on their separate
merits. The third category, concept/theme,
is in some ways a
      contradiction--the more creative artists get, the less some judges think they
are adhering to the theme. The original name
for the category was
      "creative interpretation of theme". How you interpret this category--as reward
for creativity or punishment for failing to
stick to the topic--is up to
      you. 
[3.11] What do the ratings mean? 
      They are an absolute scale of merit, with 1 being the lowest, poorest rating, 10
or 11 being average, and 20 being the
highest, best rating an
      image can receive. Each judge must decide how to award merit, but experience has
shown that sticking close to average for
most images gives
      you more room to reward those "Wow!" images, and to educate those morons who
just don't get it. 



-- 
http://www.flash.net/~djconnel/


Post a reply to this message

<<< Previous 10 Messages Goto Initial 10 Messages

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.