I have an idea which may help a little bit to avoid the so-called
category cross-contamination.
  A short recapitulation: There is a little problem in the voting of the
irtc images (and perhaps animations too): There are three different
categories to vote, but they are usually used as a general "how stunning
image this is"-vote, ie: if an image is stunningly beautiful/photorelistic
or whatever, it will probably be scored high in each category, no matter
if it isn't very good on each one of those.

  The idea: After voting each category, voters have to write a short
description about why they gave that score in that category. The votes
and description are published with each image (as the comments pages are).

  Perhaps this way people may think a bit more on each category.
  For example:

  "Wow! This is an astonishing image! Wonderful! Great! All that detail!
Let's vote."

  Tech: 20. Great image! So photorealistic! The amount of detail is
astonishing. etc etc...

  Art: 20. The image is beautiful. It's... er... well...
hmm...
  Art: 18. The image is beautiful, but perhaps a little bit unimaginative...
hmmmm... actually...
  Art: 16. The image is rather beautiful, but a little bit unimaginative.
Although it's technically great, it looks more like an architecture book
photograph, not like a painting. etc etc...

  Concept: 16. .... well, er...
better:
  Concept: 10. This image doesn't fit very well in the topic. It really
isn't absolutely out-of-topic, but I don't think it represents the topic
very well. Also there is a total lack of originality: just a boring
building with nothing special in it. etc etc...


  Just an idea.

-- 
main(i){char*_="BdsyFBThhHFBThhHFRz]NFTITQF|DJIFHQhhF";while(i=
*_++)for(;i>1;printf("%s",i-70?i&1?"[]":" ":(i=0,"\n")),i/=2);} /*- Warp. -*/

Post a reply to this message

Nieminen Mika wrote:
> 
>   I have an idea which may help a little bit to avoid the so-called
> category cross-contamination.
>   A short recapitulation: There is a little problem in the voting of the
> irtc images (and perhaps animations too): There are three different
> categories to vote, but they are usually used as a general "how stunning
> image this is"-vote, ie: if an image is stunningly beautiful/photorelistic
> or whatever, it will probably be scored high in each category, no matter
> if it isn't very good on each one of those.
> 
>   The idea: After voting each category, voters have to write a short
> description about why they gave that score in that category. The votes
> and description are published with each image (as the comments pages are).
> 
>   Perhaps this way people may think a bit more on each category.
>   For example:
> 
>   "Wow! This is an astonishing image! Wonderful! Great! All that detail!
> Let's vote."
> 
>   Tech: 20. Great image! So photorealistic! The amount of detail is
> astonishing. etc etc...
> 
>   Art: 20. The image is beautiful. It's... er... well...
> hmm...
>   Art: 18. The image is beautiful, but perhaps a little bit unimaginative...
> hmmmm... actually...
>   Art: 16. The image is rather beautiful, but a little bit unimaginative.
> Although it's technically great, it looks more like an architecture book
> photograph, not like a painting. etc etc...
> 
>   Concept: 16. .... well, er...
> better:
>   Concept: 10. This image doesn't fit very well in the topic. It really
> isn't absolutely out-of-topic, but I don't think it represents the topic
> very well. Also there is a total lack of originality: just a boring
> building with nothing special in it. etc etc...
> 
>   Just an idea.
> 
> --
> main(i){char*_="BdsyFBThhHFBThhHFRz]NFTITQF|DJIFHQhhF";while(i=
> *_++)for(;i>1;printf("%s",i-70?i&1?"[]":" ":(i=0,"\n")),i/=2);} /*- Warp. -*/


I think this might help, but I see a problem arising here. It takes a
lot of time looking through the entries carefully and then voting
carefully. I guess this is also true for most of the voters. When I
checked the voting results I saw that only 50 (!!!) votes had been made
and only 45 (!!!) complete votes had been made. This means that not even
50% of the entrants did vote! If the workload for voting increases the
number of voters may drop to a level which may not be representative
anymore. 

Maybe the numbers given in the voting form can be precised by added text
like:



artistic 20: Perfect arrangement of objects and colors...
.
.
.
artistic 10: This picture hardly shows some several flaws in the
arrangement.
.
.
.
artistic  1: This picture has absolutely no artistic merit

This method has the advantage that it doesn't slow down the voting but
it makes people more conscious what they are voting and it might even
help voting newbies.

Just my two cents.

Marc
 
-- 
Marc Schimmler

Post a reply to this message

Nieminen Mika wrote:

>   I have an idea which may help a little bit to avoid the so-called
> category cross-contamination.
>   A short recapitulation: There is a little problem in the voting of the
> irtc images (and perhaps animations too): There are three different
> categories to vote, but they are usually used as a general "how stunning
> image this is"-vote, ie: if an image is stunningly beautiful/photorelistic
> or whatever, it will probably be scored high in each category, no matter
> if it isn't very good on each one of those.
>
>   The idea: After voting each category, voters have to write a short
> description about why they gave that score in that category. The votes
> and description are published with each image (as the comments pages are).
>
>   Perhaps this way people may think a bit more on each category.
>   For example:

 <snipped good stuff>

Just to play the devil's advocate here...

There is merit in what you say but I see a possible fault in your logic.
You are asking people to rationalize a decision they have made for
each catagory and once they have made that decision for what ever
reason it was justifiable to them at the time. There may be some people
shooting from the hip on the way they vote but when pressured to
justify there decision pride will force them to rationalize that desicion
in favor of their voting scores. It may only complicate the voting
process without actually adding any real benefits to it.

Or as you say maybe it will force them into thinking about what they
are doing. Unfortunately human nature favors my former arguments.


Art critics are a fickle bunch and you may quote me on that.

--
Ken Tyler

tyl### [at] pacbellnet

Post a reply to this message

That kind of evaluation will not work because it is vague.
I would like to see:
- Change the rules on voting. eg, Vote. Send your image, register,
vote and you will be IN the contest . Only if you vote, there are a lot of
programs
that make just that, if not, do them.
- Why don't use a simple scale? What is the problem with a scale?
Very simple:  The evaluator has to know What to evaluate and how,  points
are
non important, because they can always be translated into the 20.
-I am interested in a scale - I am making one* and I would like you to help
me-. Who is eager to establish what 'exact' parameters are needed to
evaluate the technical part? Will you please send me your check points? No
matter
how many points you assign; if 100 points are what you are used to, will be
OK.
- Maybe CSG use, loops and so on,  modeling (does that scores or not, does
that is completely neccessary for the image to be what it is?, are we
evaluating the image as a POVimage or a raytraced image,  which are the same
but are not the same.etc
 Thanks in advance,

Marjorie Graterol

* The scale is for my use primarily, so it will be imperfect, but it is
something to start with. I refuse to enter in the next round and then saying
the same thing all over again. I will publish it in my home home page once I
finish them ( the scale AND the page redesign). I'll post the link  only to
the scale.

>
>Or as you say maybe it will force them into thinking about what they
>are doing. Unfortunately human nature favors my former arguments.
>
>
>Art critics are a fickle bunch and you may quote me on that.
>
>--
>Ken Tyler
>
>tyl### [at] pacbellnet
>

Post a reply to this message

Pedro Graterol wrote:

>  Thanks in advance,
>
> Marjorie Graterol
>
> * The scale is for my use primarily, so it will be imperfect, but it is
> something to start with. I refuse to enter in the next round and then saying
> the same thing all over again. I will publish it in my home home page once I
> finish them ( the scale AND the page redesign). I'll post the link  only to
> the scale.

Perhaps it is too vauge but your proposed method of evaluation by points
is doomed to failure (maybe). There are times that technical can achieve
high points without having to use every feature in the program. The technical
isssue is related, quite closely, to the other point groups, theme and composition.

For a good example look in the povray.binaries.images group at
the image "You know you have been...". This image is simplicity
in itself but the message it conveys and the artistry used to convey
that message are exemplary. In the right round I would give it fairly
high points in all categories just because it comes together so well.

If you want a grading of which povray features are most likely to be
used by a beginner and which by the more advanced user the I will
try to make a simple, prioritized, list for you. Even that will be a
subjective list because what may be difficult for me may be a simple
thing to someone else. I could only offer it as an opinionated bias.

--
Ken Tyler

tyl### [at] pacbellnet

Post a reply to this message

Ken wrote in message <36A35560.EA5AC0B4@pacbell.net>...
>Perhaps it is too vauge but your proposed method of evaluation by points
>is doomed to failure (maybe). There are times that technical can achieve
>high points without having to use every feature in the program. The
technical
>isssue is related, quite closely, to the other point groups, theme and
composition.


I don't agree with that because the whole constest is then doomed. It is
very easy to convert 100's to 20. What I am asking for is a way to be
objective. This, again, is not that easy but can be done and it is done in
every work. The simplest way is to say yes or not, e.g, You drive or not,
otherwise you crash (in conditions to drive a car) period. The other way is
to
evaluate how yo do that, assuming of course you passed the first one.  You
can list all the aspects you want and assign any value. The next step is
prioritizing, what goes first. e.g, I cannot drive if I cannot turn the
engine on. Dumb? Not. It happens.
>
>For a good example look in the povray.binaries.images group at
>the image "You know you have been...". This image is simplicity
>in itself but the message it conveys and the artistry used to convey
>that message are exemplary. In the right round I would give it fairly
>high points in all categories just because it comes together so well.

I do not have any problem with images themselves., in fact,  I usually
under-evaluate them. In the same way it is very easy for you to establish
the technical things and how they go together., in the same way I "look at"
images.  And I was not asking for your help in this aspect., just because I
know the field.

>
>If you want a grading of which povray features are most likely to be
>used by a beginner and which by the more advanced user the I will
>try to make a simple, prioritized, list for you. Even that will be a
>subjective list because what may be difficult for me may be a simple
>thing to someone else. I could only offer it as an opinionated bias.


I was asking for those points because it is irresponsible for me to evaluate
something I do not know enough., but this is something that happens the
other way around, simply put, I am not a programmer and you are not and
artist and viceversa.

And yes, I have many options:

1- Enter in the contest and not judging  -letting others that really know to
evaluate my work -
2- Ask for categories inside the contest, beginner, advanced, out of this
world, etc.
3- Be very aware of the nature of judges, and what they evaluate.
4- Do my homework without asking for help, stop whinning and go ahead no
matter what.

Obviuosly, 4.
Thanks

Marjorie Graterol



>

Post a reply to this message

Ken <tyl### [at] pacbellnet> wrote:
: Just to play the devil's advocate here...

  Why that doesn't surprise me?-)

  Your point of view is also very justifiable and probably true.
  But anyways, there's another thing the method I presented might help and
which I forget to mention in my article:
  Often people score very low in each category if the image is very ugly,
but if they have to state arguments for their vote, then they really have
to think why they are giving so low scores, and perhaps they will notice
that scoring very low in each category is not very fair, since, although
very ugly, the image may have a very good idea which represents very well
the topic, etc.
  Perhaps (this is only a dream...) this way votes would be more fair.

-- 
main(i){char*_="BdsyFBThhHFBThhHFRz]NFTITQF|DJIFHQhhF";while(i=
*_++)for(;i>1;printf("%s",i-70?i&1?"[]":" ":(i=0,"\n")),i/=2);} /*- Warp. -*/

Post a reply to this message

just curious :
could you cite me an image (even from past rounds) that
deserves a vote like 20-10-12 or 10-9-16 or anything like that ?
I don't remember seeing one, even after 16 months of IRTC.

Cheers,
Fabien.

Post a reply to this message

Nieminen Mika wrote:

>   I have an idea which may help a little bit to avoid the so-called
> category cross-contamination.

	My zwei Pfennig in this comes from scoring many proposals for
the US government. 

	This is the first time I have read about the scoring, being a
newcomer and all. 20 levels? Go to 6 levels, 0-5, with 3 as
average and 0 as [expletive deleted.] Or outstanding, above
average, average, below average, poor, expletive. 

	Unless people have nothing better to do and have no shame they
are not going to participate in the first place unless they have
done something noteworthy compared to what they have seen on the
web. All of us newcomers hunt for everything we can find and
compare, borrow, whatever. So users have a good idea what is
going on and if they are competative before spending the time. 

	Average becomes the state of the art in comparable renderings.
As in sports, the average goes up every year as does outstanding
as in 70 homeruns last year. Doing better than Babe Ruth is no
long for the record book. 

	1) This gets rid of the fine distinctions that cloud judgement.
It is either outstanding or it is not. 

	2) Evaluators tend to avoid 5 as much as 0. It lets things stand
out like a great technical accomplishment without artistic merit.
For example, the first realistic rendering of X used in a manner
that is almost not on topic and being nothing but the object in a
featureless background. People are not likely to be that extreme
but without a 4.4 available it causes differentiation. 

	3) It makes judging much easier and can get more judges. It is
much easier to judge between, ho-hum, impressive and if your jaw
drops on seeing it. 

	And from that government experience I am aware that categories
tend to go from 3 to 5 to 10 to 20 over the years and then
someone starts over again at 3. It is quite cyclic.

Post a reply to this message

This ain't high diving.

Any ol' fool can throw a bunch of sloppy CSG and isosurface patches and fail
miserably for technical merit in my book.  I think that Annig's earlier post
talked about "appropriate use" of technology.   I have rated images low for
being an obtuse showcasing of a raytracing program's capability.

An entry could use nothing but boxes, cylinders, and one plane, and deserve a
higher rating than another image which used heightfields to texture a
building.   Cross-contamination is one way to describe a vote you disagree with.
Consider the entry a few rounds ago with bald, pale, hairless, identical
(Poser2) cavemen.  I found that detail offensive to its artistry: who'd want
that entry hanging on a wall?  It was offensive to technical merit: it showed
careless use of a canned model.  It was offensive to concept:  WHY ON EARTH
would a group of cavemen be cloned and bald?  What kind of message is the author
trying to convey:  time travel? loss of identity in cults?


Pedro Graterol wrote:

> -I am interested in a scale - I am making one* and I would like you to help
> me-. Who is eager to establish what 'exact' parameters are needed to
> evaluate the technical part? Will you please send me your check points? No
> matter
> how many points you assign; if 100 points are what you are used to, will be
> OK.
> - Maybe CSG use, loops and so on,  modeling (does that scores or not, does
> that is completely neccessary for the image to be what it is?, are we
> evaluating the image as a POVimage or a raytraced image,  which are the same
> but are not the same.etc
>  Thanks in advance,
>
> Marjorie Graterol

Post a reply to this message