 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
As might have been menitioned elsewhere, the problem with one category is
thematic deviation (yes I've done it). But what about the fact that someone
has a really great idea for their image, and just didn't develop that idea.
(Again, I've done this). If there was one category, there would be no way to
honor that person for a great idea, but a lousy image.
In my thoughts, I think the categories should be the following, because these
are distinct, but still tie together with each other:
-Concept/originality
-Technical Merit
-Impressiveness
Concept/originality is basically how the topic is developed in the image. It
shows how well through the image was thought. e.g. In one of the Stills rounds,
called "Alien Worlds," there were some beautiful Star Trek matte paintings
rendered, but that's not really original as an idea. So, Plus for the
artistic, minus for the orignality
Technical Merit is basically how the user approached the image s/he made. Did
s/he use some impressive code to produce a high degree of photo-realism, or did
he simple put a few spheres next to each other, and just image-map them.
Basically, it's how much work the user put in to achieve his goal.
And Impressiveness is basically just how well the image pulls off as a good
image. The more it makes someone go "Wow" and the more it looks good, might be
the scale. That doesn't just mean that photo-realism is impressiveness. I
mean, I get more "wowed" from Pixar's Toy Story, than I do from Harry Potter's
Magic Effects. It just is how good the image looks.
By all means, don't quote me on saying that's how it should be. I personally
think this would work well, with a bit of tweaking here or there, but that
system seems good to me.
And to David, if you use this system, or something very similar, by all means,
you may quote me in the voting guidelines, or rules, etc.
Any other ideas on those categories?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
One of the most offensive scores I got on an IRTC submission was 10 10 10. And
on that round, when I voted on others, I made sure my vote expressed exactly
what the artist were doing well (artistic, technical, theme). This basically
means that everyone else gave me 10 10 10 for the average to be 10 10 10. No
one put any thought into the vote.
For a vote to be meaningful, it has to COST the voter something. In the old
voting system, one could also give 20 20 20 to one's favorite, and 1 1 1 to all
the ones one didn't like. Others might give out a strict normalized
distribution around ten, and reserve 20's for once a lifetime beauties.
Average up these two voters' votes, and you're throwing away the careful
voter's analysis.
For the Short Code Competition, I came up with a voting scheme where you cite
your favorite six entries, and then gave 6 points to the favorite, 5 to their
2ndmost, and so on. In this way, everyone's opinion is weighted the same.
There are more complicated schemes where folks have an IRTC Dollar to give
away, and they can divide it up any way they want-- all to one or a penny to
100 entries-- I say this is inherently fair, as every voter gets a dollar.
Analogy: My local newspaper once spent a lot of time and effort into a reader
vote on which comics to keep. They let everyone vote yes or no on keeping
every comic. (Note there's no COST to vote here!) The comic "Doonesbury" came
in about last, but they never dropped it. My hunch is they knew that 55%
didn't like it but maybe 10% were passionate about it. Because the poll didn't
measure user passion, the paper ended up ignoring the results.
As far as WHO votes, I'd be more inclined to have real people vote than male
nerds who use freeware graphics programs. (DISCLAIMER: I'm one of the latter.)
Maybe it's impractical to line up a set of artistic, educated, and/or
interesting people, but that would be the coolest.
David Buck <dav### [at] simberon com> wrote:
> I'd like to find a nice way to prevent or reduce voting anomalies in the
> new IRTC web site. I don't want people to be able to vote multiple
> times in one contest and skew the votes.
>
> So I'd like to know if you have any suggestions on how to do this.
>
> 1) How do we accept new voters?
> - automatically with e-mail confirmation?
> - only with admin approval?
> - other ideas?
>
> 2) Should submitters be allowed to vote in the round they submit to?
>
> I intend to allow voters to rate each image in 4 categories
> - overall
> - technical merit
> - artistic merit
> - concept and interpretation of theme
>
> Each rating is on a scale from 0 to 10.
>
> 3) How should the scores be combined
>
> 4) Should we allow partial voting?
> - by this I mean rating some images but not all
> - I have no assurance that they even looked at all of them
> - if you rated the first 5 pictures but none of the others,
> should any of your ratings be considered?
>
> Your thoughts would be appreciated.
>
> Thanks
> David Buck
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On Mon, 10 Mar 2008 06:39:00 -0700, Randal L. Schwartz wrote:
> You could let each person select a "lo" and "hi" point of their own
> scale, and then normalize the values to a floating point from 0.0 to
> 1.0.
Ewww.
Rather than do that, why not just let people pick from a 1-9 scale, and
if they want it more coarse, they can ignore every other digit?
Letting people pick their own scale will create problems - someone thinks
they picked a scale of 3, but the app saw it as a scale of 9, so instead
of giving the top rating, they give one of 33%?
No way....
Jim
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On Mon, 10 Mar 2008 08:49:22 -0500, Warp wrote:
> Does anyone have any good idea about how to avoid cross-contamination
> of categories?
If you get enough votes, cross-contamination of categories will factor
itself out in the averages.
Jim
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
47d2ce11$1@news.povray.org...
> I intend to allow voters to rate each image in 4 categories
> - overall
> - technical merit
> - artistic merit
> - concept and interpretation of theme
It should be noted that with the IRTC, the 3 notes (techical, artistic,
concept) were always extremely correlated. In the 2006 rounds the
correlation coefficients between these notes were in the 0.92-0.96 range:
it's basically the same note (correlation coefficients run between 0 and 1
where 0 = no correlation). One can run a regression and predict one note
with another with a small margin of error. I did the same calculations in
the past, with the same results. In a nutshell, folks who are smart enough
to understand a concept and come up with a good idea also have decent
artistic and technical skills. Now that's really surprising ;)
There are occasional outliers of course - for instance images with a
moderately good artistic note and so-so concept - but on the whole having
several voting categories is more intellectually satisfying than
statistically meaningful.
G.
--
*****************************
http://www.oyonale.com
*****************************
- Graphic experiments
- POV-Ray, Cinema 4D and Poser computer images
- Posters
>
> Each rating is on a scale from 0 to 10.
>
> 3) How should the scores be combined
>
> 4) Should we allow partial voting?
> - by this I mean rating some images but not all
> - I have no assurance that they even looked at all of them
> - if you rated the first 5 pictures but none of the others,
> should any of your ratings be considered?
>
> Your thoughts would be appreciated.
>
> Thanks
> David Buck
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
"gregjohn" <pte### [at] yahoocom> wrote:
> One of the most offensive scores I got on an IRTC submission was 10 10 10.
>...This basically means that ...No one put any thought into the vote.
>
> For a vote to be meaningful, it has to COST the voter something. ...
>
> For the Short Code Competition, I came up with a voting scheme where you cite
> your favorite six entries...
.....
> As far as WHO votes, I'd be more inclined to have real people vote than male
> nerds who use freeware graphics programs. (DISCLAIMER: I'm one of the latter.)
> Maybe it's impractical to line up a set of artistic, educated, and/or
> interesting people, but that would be the coolest.
Although I'm very sympathetic to the sentiments that you express here and
recognize that they are thoughtful and well-intentioned, I nonetheless feel
compelled to mention a few problems that I see with the particulars:
Systems under which a judge votes only for a few favorites tend to leave
everyone except a few elites out in the cold with no constructive feedback.
This bodes poorly for developing the community and building up the skills of
the up and coming. I guess this really comes down to a question of goals and
priorities. *ARE* we interested in building community and encouraging
Also, sadly, one of the things that must be considered is how a system might be
abused by those who do not share your good intentions and desire for fairness.
If I'm more interested in influencing the outcome in favor of my best friend or
my own little clique than in judging the entries fairly, I can do much more
damage by concentrating all 100 of my points than I can by being just one voter
with a skewed motive. (I freely acknowledge that NO system solves the ulterior
motive problem entirely, but voting only for favorites exacerbates it. )
effort. Not only should voters be required to vote on ALL entries, but also to
leave at least one coherent comment on each entry AND to explain why any image
was given either the highest or lowest possible rating. To those who would
suggest that this is asking too much of judges, I would reply that it's nothing
compared to the effort of preparing a serious entry.
As for the WHO of voting, (and this is merely personal feeling with no real
objective logic behind it) I share your desire to get feedback from both inside
and outside the world of those who produce ray trace images, but I would require
don't give a rodent's rectum how people who rush home from their marketing jobs
would rate my work. Also, a significant percentage of the judging will have to
about how one can improve.
Respectfully,
Mike C.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
I like the idea of forcing people to leave comments. I usually try to do this,
so as to give others a chance to see why I felt the way I did. I remember that
once,I even forgot something in the comment, and left an anonymous comment just
to add something.
Back then, I was an amateur Ray-tracer, and was soo hoping for comments on
improving my images. In both stills competitions I entered, I recieved one
comment. However, Animations were kinder, in giving me about 5 comments.
I think some people don't realize that there are amateurs out there who enter
partly for fun, and partly for just trying to get better at something they
really love. When I got no comments, I thought it was because no one cared for
such low images. Admittingly, one did get last place, but I never got to figure
out why it did. I deduced most of it over a years worth of developing skills.
So, yeah, I agree that comments should be highly recommended, if not forced.
Someone who takes the time to write out comments for every entry, has
definitely thought each image through, and also hopefully graded each likewise.
As for other things, points need to be something the whole world understands. I
definitely think there needs to be a guide as to what each rating means(they
used to do that for the 1, 10, and 20 marks). Points mean a lot about how well
an image really is, and they themself can tell a contestant what there image
really is. So, that's why I'm supporting categories.
David, have you come to some decisions on what to do, or are you still waiting
out?
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Nimish Ajmani wrote:
> David, have you come to some decisions on what to do, or are you still waiting
> out?
I've been following all the discussions here and putting together what I
feel are the best recommendations. I'll put together a summary of those
later this evening and post them here.
Thanks to everyone for their feedback - it's very helpful and has
changed my approach to the design of the site.
David Buck
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
> Thanks to everyone for their feedback - it's very helpful and has
> changed my approach to the design of the site.
I think I speak for everyone when I say not a problem.
--Nimish
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>
> There are occasional outliers of course - for instance images with a
> moderately good artistic note and so-so concept - but on the whole having
> several voting categories is more intellectually satisfying than
> statistically meaningful.
I wholeheartedly agree.
As a numerous-times IRTC participant, I've been mostly annoyed
by these useless 3 categories, and, personnally, I had the habit
to put the 3 same notes each time.
Advice : get a general ranking with a single note, and use a small jury
to attribute artistic, technical and concept merits. Makes more sense.
Fabien.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|
