"David Buck" <dav### [at] simberoncom> schreef in bericht 
news:47d5f520$1@news.povray.org...
>    - Registration will capture home address and phone number
>       - this helps prevent fraud
>       - allows IRTC admins at their discretion to award prizes
>       - registration information is only available to IRTC admins
>

This might be a minor/major issue in terms of privacy regulations. The 
information therefore should really be secured.

Thomas

Post a reply to this message

Warp wrote:
>   I think we should consider some extraordinary circumstances and what
> to do if they happen (let's hope they don't, but...)
> 
>   What will be done if the amount of submissions is ridiculously low?
> For example, let's assume that two people submit for a given round.

Then those two people will sweep the prizes ;)  Seriously, we can throw 
a war, but we can't guarantee that anyone will come fight...

>   What will be done if the amount of votes is too low? Let's assume that
> in a round of 20 images only 2 people vote.
> 
>   Where will the line be drawn?

Supposedly that's why there are panel judges.

-- 
...Ben Chambers
www.pacificwebguy.com

Post a reply to this message

Thanks for the proposal David.

I have two comments that I'd like to make:

First, given Gilles charts, does it still make sense to have 3 categories?
Or if we do have more than one category, could we change them so they are
not so closely related, and only grade the contest on one of them? (i.e.
the other one(s) are useful for ranking in terms of that category but don't
influence the outcome).

Several categories come to mind along those lines:

  o 'eye candy' factor
  o hof-worthy [this would only be useful for POV-Ray images of course]
  o educational value [from description or source code]

By 'educational value' I don't mean socially educating but value in
educating users of the tool in question, or of 3d graphics in particular.
For example a person who regularly enters, doesn't win, but over time
accumulates a particularly high average 'educational value' score could be
pointed out for special honor at some point, or something ... basically
these other categories are a method of tracking 'interesting' information
that may be one day useful to the admins, as well as allowing those viewing
the rounds to sort by that category if they so wish.

Secondly, for each round in which more than (say) 10 people vote (including
panel judges, and hopefully there will be at least several of those), I'd
like to suggest that the N highest and lowest votes for each image are not
counted. For # of voters 10-15, N might be 1. For 16-30, N could be 2, and
so forth. So if we have 30 voters, the two highest and two lowest votes for
each image are disregarded - so the final score is calculated from the
remaining 26 votes.

The intent of this is to attempt to offset the influence of votes that are
well outside the norm for an image. I recognize that it's rather crude, and
a better approach to looking for voting anamolies would be to e.g. view a
standard deviation graph or some such, but that's something a statistician
would be better qualified to talk about, rather than me.

IIRC (not 100% sure but I'm fairly positive - I'd have to check the code to
be 100%), when calculating the POVCOMP votes, we did exclude the highest
and lowest for each image (even though all votes came from a panel).

-- Chris

Post a reply to this message

David Buck wrote:

>    - Registration will capture home address and phone number

I don't see this as a problem but is a phone number really useful
for an international internet-based competion? It's usually email
for communication and login id, and sometimes home address for
verification and shipping purposes.

I assume registration will also capture the real name,
although that wasn't explicitely mentioned.

>    - Votes would be on a scale of 1 to 10
>       - A vote of 0 means the voter didn't rank the image

The use of the 0 seems a bit implementation oriented - oh,
let's store this information in an integer variable :-P

I think the interface should be able to provide something
like "-" or "unrated" to make it more clear. You could then
also encode it that as -1 and allow 0 as a vote.

>    - Every voter must provide a comment on each image they vote on

I agree that comments are very important, but I'm not sure if
enforced comments are helpful. It may just lead to a lot of "nice
image, I like it" responses for images where you wouldn't have
left a comment because you just didn't see any particular
problem or outstanding feature.

Furthermore, voting is a responsibility, but also a service
provided on a voluntary basis, so it feels awkward to enforce
it. It might backfire by deterring people from voting.

Also, it's obviously not something which automated system are
very adept at checking, and even administrators may be on thin
ice when disqualifying because they feel the comments are not
good enough (not talking about offensive comments, of course).

I'd opt for a strong recommendation to leave comments for as
many images as possible, enforcing comments for extreme votes
(you can't very well rate an image as 1,2,9 or 10 without
having something to say about it), and possibly requiring
a certain minimum number of comments (like 20%?).

A final technical note: It would be nice if the system was
capable of storing incomplete votes. I generally have a very
low trust level in browser compatibility issues when it comes
to filling out and submitting very long and tedious forms ;)

BTW, shouldn't these matters be discussed in irtc.general,
where they might also be seen by non-pov participants? Or
does new IRTC target POV-Ray alone?

Post a reply to this message

David Buck wrote:
> Nicolas Alvarez wrote:
> It would probably be a wise move in any event.  If you were to win and 
> the IRTC admins decide to send out prizes (T-shirts, mugs, posters, etc) 
> they'd need to know where to send them to.

Of course we could always ask at the time. Honestly, provision of an
address is not deterrent for someone who wants to cheat since they'd just
make one up. However, that said, I think a contact address should be
provided, but for a different reason.

Do you know how *frustrating* it is to have someone contact you to ask
permission to use an IRTC image for a good cause, and to have to refuse
because you can't contact the author?

I do [:-(

I get IRTC admin mail, and have sometimes spent excessive time trying to
help (not that I am obliged, but sometimes if it's a good cause I want to)
find an entrant who used an alias and a free email address (or just simply
an old address - recall some entries are more than ten years old). In
today's world of spam-obliterated email accounts there's a good chance that
if I don't have some other means of cross-referencing them I won't be able
to find them.

For that reason, I'd prefer that registration provide some information (not
to be released to the public) that would help the competition locate an
entrant at some point in the future, if their last known email address
bounces or goes unanswered. This could be a phone number, postal address,
parents address, or really anything - voluntarily supplied for this purpose.

I'd also suggest that if entrants have a 'private' email address that they
don't give out to the public but don't mind giving to the admins, that they
do so as well, via the registration system. One that they are less likely
to change, for example.

Here's an example of why it's good to have additional contact information:
while it doesn't apply to the IRTC as we won't be awarding prizes as a
regular thing, it's nevertheless worth mentioning.

There was one entry in POVCOMP where the author had done everything other
than push the 'final button' (as it were) that clicks an image over from
being a 'work in progress' (which he could update or edit), and makes it a
'final' (locked) entry. The action of doing so also requires them to agree
to some rules and copyright stuff, so it's important and can't be done on
their behalf by an admin.

This entry was - in my personal opinion - very worthwhile and would (again
IMO) certainly have won one of the prizes. The auto-reminder system wrote
the author to tell him about the problem. I also manually emailed him more
than once (perhaps even three times). No reply. So the competition closed
without his entry, and he missed out on a prize.

Later on, when the results were announced, he turned up in the newsgroup
expressing surprise that his entry wasn't even in the competition. He
*then* found that his spam filter had snarfed all of the competition's mail
to him, and it was in fact still sitting unread in his spam folder.

Of course having a second email address won't necessarily fix that, but it
does point out the fact that if we want to contact you, it's not in these
days enough to rely on email. It's just not reliable anymore due to spam.

-- Chris

Post a reply to this message

>     - Registration will capture home address and phone number
>
Home address?
Hm, I'm affraid, this can hold back people from registering...

> David Buck

Slawek
-- 
  ________ When the lusers ring! When the pager beeps! And at 3am they call.
_/ __/ __/ I simply remember I'm the BOFH, And rm -r... them all.(Steve Shipway)
 \__ \__ \_______________________________________________________________

Post a reply to this message

Further to my other comments, I'd like to add this:

It would be neat if in the image submission process, the entrant could (via
some click-through legalese) agree that the competition admins may, at
their discretion, choose to grant third-party requests for non-commercial
use of their entry if reasonable attempts to contact them fail*. Or have it
the other way around; make this a default condition of registration/entry
and require them to opt out of it if they do not wish to do so.

FWIW, permission for use was by far the most common request we received on
the IRTC admin mailing list. It would be nice if there was a neat way to
handle this, given that in my experience most entrants were happy for their
work to be used in a reasonable non-commercial way provided they are credited.

-- Chris

* or even let them specify, if they wish, that we don't need to attempt
  to contact them.

Post a reply to this message

Fa3ien wrote:

>>    - Ranking will be in 3 categories
> 
> Seen Gille's latest post on the matter ?
> Seen the graphs ?
> 
> Why keep the voting process 3 times longer than necessary ?
> 
> Fabien.

Yes, I've seen the post and the graphs.  I thought carefully about the 
charts and the graphs.  Here's my problem:  Currently we have 6 awards - 
1st, 2nd, 3rd, Technical, Artistic and Concept.  Do we continue to give 
6 awards?  We can reduce to one ranking per picture if we drop the last 
3 awards. If you want to provide the last three (automated) awards, how 
do you propose we award them with only one rank?

In terms of voting time, I'll be using a star-rating control where you 
click on one of 10 stars to vote.  Since I'm proposing that we require 
voters to write a comment for each submission, typing the comments will 
take far longer than the voting itself.

Do you have a proposal on how many awards to give and how to calculate them?

I'm looking for constructive comments on the voting system.  Please, if 
you're going to criticize something I proposed please offer your version 
of an alternative.  As I said, I can't satisfy everyone and I'm doing my 
best to get public input while still keeping the spirit of the competition.

David

Post a reply to this message

Christian Froeschlin wrote:
> I'd opt for a strong recommendation to leave comments for as
> many images as possible, enforcing comments for extreme votes
> (you can't very well rate an image as 1,2,9 or 10 without
> having something to say about it), and possibly requiring
> a certain minimum number of comments (like 20%?).

This proposal has a lot of merit IMO.

-- Chris

Post a reply to this message

le message de news: 47d65931@news.povray.org...
> Thanks for the proposal David.
>
>  o 'eye candy' factor
>  o hof-worthy [this would only be useful for POV-Ray images of course]
>  o educational value [from description or source code]

That seems more relevant than the old category system. The "education value" 
sounds like karma points ;) That would make possible, for instance, to 
reward people for making the source code available (or models and textures 
provided in compatible formats) while keeping it separate from the judgement 
itself.

> The intent of this is to attempt to offset the influence of votes that are
> well outside the norm for an image. I recognize that it's rather crude, 
> and
> a better approach to looking for voting anamolies would be to e.g. view a
> standard deviation graph or some such, but that's something a statistician
> would be better qualified to talk about, rather than me.

It's not crude at all but a well-recognized statistical procedure.
See http://en.wikipedia.org/wiki/Truncated_mean
One could remove the top and bottom 5% of the votes for instance.

G.


-- 
**********************
http://www.oyonale.com
**********************
- Graphic experiments
- POV-Ray, Cinema 4D and Poser computer art
- Posters

Post a reply to this message