"Thorsten Froehlich" <tho### [at] trfde> wrote in message
news:3dd3a7ab@news.povray.org...
> In article <3dd38dbe$1@news.povray.org> , "Tom Melly" <tom### [at] tomandlucouk>
> wrote:
>
>
> You may have noticed that I gave _two_ ways to do it.  Either even and odd
> frames or splitting in the middle.
>

But neither of these work if, in the case of even and odd, alternate images take
significantly different amount of times to render (odd frames use media, even
don't), and in the case of splitting in the middle, if the second half of the
sequence is significantly slower or faster than the first half (or, needless to
say, if one PC is slower than the other).

The point with starting at opposite ends is that if the final frame takes as
long as all the other frames put together (either because it's running on a
slower machine, or just becuase of the nature of the image), then one
render-instance renders only one image, and the other render-instance renders
the rest.

I think the misunderstanding here is that a) I don't propose that each
render-instance should render the same number of frames and b) I don't propose
that either render-instance "knows" when to stop - it would be up to the user to
note that instance-a had rendered images 1-20 and that instance-b had rendered
images 30-21 and that therefore the render could now be stopped.

Nor, btw, do I propose that this would lead to an "exact" distribution - one
instance would invariably finish its final frame before the other, whatever
frame that might be.

Post a reply to this message

Ken <tyl### [at] pacbellnet> wrote:
> How can a second instance of POV-Ray possibly know exactly which frames had
> been rendered by the first instance.

  The two processes must communicate in order for them to know where to stop.
That is, when one process has rendered a frame and is ready to start the
next frame, it asks the other process if that frame is free. If the other
process answers "yes", then it proceeds, else it stops.

  And yes, communication between two processes suffer from all the
mutual exclusion problems typical to parallel processing, and if not
well done, the two processes might start rendering the same frame. This
shouldn't be a big problem in this case, though. And besides, there are
algorithms which handle the mutual exclusion problem.

  For those who don't know, the mutual exclusion problem means that
if the access to a common variable is not locked between the "ask if
the frame is free" and "set the frame as non-free" commands, then the
other process might ask for the variable after the first process asked,
but before it could set it as non-free. Then both will get a "it's free"
value which is of course not desired.
  How to lock other processes from accessing the variable while one process
is reading and setting it is an interesting problem with interesting
algorithmical solutions.

-- 
#macro N(D)#if(D>99)cylinder{M()#local D=div(D,104);M().5,2pigment{rgb M()}}
N(D)#end#end#macro M()<mod(D,13)-6mod(div(D,13)8)-3,10>#end blob{
N(11117333955)N(4254934330)N(3900569407)N(7382340)N(3358)N(970)}//  - Warp -

Post a reply to this message

Thorsten Froehlich <tho### [at] trfde> wrote:
> So you suggest one renders the even and another the odd indexed frames?  Or
> do you suggest that the beginning and the end of a sequence take long and
> you divide them such that one renders from beginning to the middle of the
> sequence and the other from the end to the middle of the sequence?  Why
> would it have to work backwards in either case?  The speed would be the same
> regardless of sequence direction!

  No, I mean that one process starts from the beginning and the other from
the end, and when they meet at the same frame, they stop.
  (Yes, this needs communication between the two processes.)

> No, this would not distribute perfectly regardless of the image except
> trivial cases.  In fact it would be very far away from perfect distribution
> as each and every pixel can take a "random" amount of time.

  As someone else said, I was talking about distributing the rendering
effort among both processors. The idea is exactly the same as above
(ie. when the two processes meet at the same pixel, they stop).

  (In the case of rendering a single image with two processors mutual
exclusion problems don't really matter, as it's not too dangerous if
the two processes happen to render the same pixel before stopping.)

-- 
#macro N(D)#if(D>99)cylinder{M()#local D=div(D,104);M().5,2pigment{rgb M()}}
N(D)#end#end#macro M()<mod(D,13)-6mod(div(D,13)8)-3,10>#end blob{
N(11117333955)N(4254934330)N(3900569407)N(7382340)N(3358)N(970)}//  - Warp -

Post a reply to this message

In article <3dd3c432@news.povray.org> , "Tom Melly" <tom### [at] tomandlucouk> 
wrote:

>> You may have noticed that I gave _two_ ways to do it.  Either even and odd
>> frames or splitting in the middle.
>>
>
> But neither of these work if, in the case of even and odd, alternate images
> take significantly different amount of times to render (odd frames use media,
> even don't), and in the case of splitting in the middle, if the second half of
> the sequence is significantly slower or faster than the first half (or,
> needless to say, if one PC is slower than the other).

I never said they would work, I asked what Warp was talking about.  Notice
the questionmarks and the wording.

> I think the misunderstanding here is that a) I don't propose that each
> render-instance should render the same number of frames and b) I don't propose
> that either render-instance "knows" when to stop - it would be up to the user
> to note that instance-a had rendered images 1-20 and that instance-b had
> rendered images 30-21 and that therefore the render could now be stopped.

But that is not what Andrew has been looking for.  If one would know in
advance how many frame each instance will render the need to start at both
ends does simply not exist.


    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org

Post a reply to this message

"Thorsten Froehlich" <tho### [at] trfde> wrote in message
news:3dd3cc49@news.povray.org...

<snip>

Now you've got me confused. Warp proposed (or rather, he elaborated on) a
theoretical method to evenly distribute a render. You made a wrong assumption
(that the idea was for the two renders to meet at the middle frame/pixel), and
then continued to assert that the approach was "wrong", even when every effort
had been made to clear up the assumption.

I've re-read the entire thread, and, yes, I got somewhat confused about whether
you were proposing or criticising at one moment, but the point is that no one
ever proposed what you criticised - you just misunderstood what was being
proposed.

Also, the confusion on my part arose quite reasonably from this:

"You may have noticed that I gave _two_ ways to do it.  Either even and odd
frames or splitting in the middle."

Now, you don't in any way imply that you agree with me (i.e. that the two
methods won't work), you just carry on criticising Warp's proposal, and picking
up on unimportant details, such as that there may be some overlap on the final
frame.

What I'm getting at is that at no point have you said "ah, I misunderstood",
even though you evidently did.

That said, I like the proposal of checking for whether a file exists, but that
will only work where the two instances share a file-system.

Post a reply to this message

In article <3dd3d95c@news.povray.org> , "Tom Melly" <tom### [at] tomandlucouk> 
wrote:

> Now you've got me confused. Warp proposed (or rather, he elaborated on) a
> theoretical method to evenly distribute a render. You made a wrong assumption
> (that the idea was for the two renders to meet at the middle frame/pixel), and
> then continued to assert that the approach was "wrong", even when every effort
> had been made to clear up the assumption.

No, I asked which of the two methods he suggested as no communication of the
two instances could exist.  The only way it could work without communication
is that disjoint sets of frames are rendered by each instance.  So I asked
which of the two simple methods possible creating disjoint sets of frames he
is talking about.  This is really perfectly clear, at least to me (see
below), and I fail to see why you then you assume I would probably be too
inexperienced or for some other reason not know that different frames can
take a different amount of time to render (remind me, how much time do I
spend optimizing POV-Ray per day?) and go on explaining it to me.  I just
don't get it.

> I've re-read the entire thread, and, yes, I got somewhat confused about
> whether you were proposing or criticizing at one moment, but the point is that
> no one ever proposed what you criticized - you just misunderstood what was
> being proposed.

OK, so the term "So you suggest" does not make clear that it is not me
suggesting something.  Would you be so kind to reword what I said such that
these misunderstandings can be avoided in the future?  I simply do not know
a better way to express that I am talking about an assumption I make about
what someone else is saying:

"So you suggest one renders the even and another the odd indexed frames?  Or
do you suggest that the beginning and the end of a sequence take long and
you divide them such that one renders from beginning to the middle of the
sequence and the other from the end to the middle of the sequence?  Why
would it have to work backwards in either case?  The speed would be the same
regardless of sequence direction!"


    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org

Post a reply to this message

In article <3dd3d95c@news.povray.org> , "Tom Melly" <tom### [at] tomandlucouk> 
wrote:

> What I'm getting at is that at no point have you said "ah, I misunderstood",
> even though you evidently did.

No, I misunderstood absolutely nothing.  I simply asked for more details,
and then mentioned that the two ways I could possibly understand what Warp
says will both not work.

After all, you lectured me on the most simple fact possible, so obviously
you just jumped on something I said, ignored that it was a question
(questionmarks are valid in English, aren't they?), and then go on and
accuse me of misunderstanding when I ask a question?

I think I will just killfile you.  Sorry!

    Thorsten


To quote your initial reply:
> Huh? I think you're missing the point. As far as I can see, starting one
> render from the last frame and another from the first frame is the only way to
> ensure the best distribution of work. Take 16 random numbers between 1 and 100
> and place them in sequence:
>
> 34, 43,11, 76, 32, 2, 23, 73, 83, 23, 76, 63, 31,63,68,65
>
> Numbers 1 to 8 add to 294, 8 to 16 add to 472. Without knowing the render time
> of each frame prior to rendering, the only way I can ensure that both renders
> finish at the same time is to start one render at one end of the sequence, and
> the other render at the other end of the sequence.


____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org

Post a reply to this message

"Thorsten Froehlich" <tho### [at] trfde> wrote in message
news:3dd3e683@news.povray.org...

<snip>

Well, at the risk of being killfiled, this is getting silly.

Andrew asked if it was possible to start an anim render on two machines,
starting at opposite ends. You proposed that a solution would be:

>"How about taking the much simpler approach of just starting the second one
>in the middle?  Kind of more logical..." (that's your first reply to Andrew's
post)

Then Warp points out that this doesn't achieve what Andrew wants - and then,
despite the fact that both Warp and Andrew have been very specific, you come up
with two non-working interpretations of what they have said ("non-working" in
the sense that they don't achieve an optimisation of render time).

On top of this, you also state that starting a render from opposite ends does
not achieve an optimal render time:

>"Otherwise such a simple attempt of load distribution
>will simply not give you any advantage at all."

Which is plainly false. So tell me, how am I to interpret this as anything else
but that you've misunderstood what is being proposed?

Aditionally, you state that:

>"But that is not what Andrew has been looking for.  If one would know in
>advance how many frame each instance will render the need to start at both
>ends does simply not exist."

Well, imho, Andrew knew exactly what he was looking for (from his original
post):

>"Forgive me if this has been asked before, but is it possible, when rendering
on 2 machines, to
>have the second machine start at the last frame, and work towards frame 1, so
both machines
>meet each other at some point?"

Where does he state that he needs to know how many frames each instance will
render?

Now, I'm sorry if I've offended by my daring to try and clarify, but you
seem/seemed confused about what was being proposed - I am well aware of your
contribution to pov, but I am also aware that developers don't tend to do much
complex scene creation (which is not a criticism - those that can, program,
those that can't write scenes;), so what might seem an obstacle to you (lack of
co-ordination between the two instances), might not seem an obstace to someone
rendering a complex scene.

Andrews suggestion, for example, would be a boon for me - if you apply Warp's
varient that you could apply it to a still image, and have two renders render
the scene - one from top left to bottom right, the other from bottom right to
top left.

I could render the image in two unequal halves on my home and work machine. It
might seem a problem that there is no co-ordination between the two processes,
but if the render takes, say 7 days, this is not really an issue, since I will
have a chance to check the progress of each render and choose an appropriate
time to stop them.

Now, apart from my terrible crime of trying to clear up what seemed to be a
misinterpretation on your part, my only other sin was that I became confused
between what you were proposing as a solution and what you were criticising as a
solution (a criticism based on your misinterpretation of what was being said).
Given that your proposal:

>"How about taking the much simpler approach of just starting the second one
>in the middle?  Kind of more logical..." (that's your first reply to Andrew's
post)

and your criticism invoke broadly similiar methods (with broadly similiar
problems), I don't see that my confusion is so shocking.

Now as I said, I'm sorry if I offended, but to be frank if you are going to take
offense when you quite clearly misunderstood something, then I'm probably better
off being kill-filed....

Post a reply to this message

Andrew Cocker wrote:

> The user would cancel one or both instances when he/she sees that the 'middle' point
has
> passed, and there are no more frames to be rendered. If the 'middle' point is passed
and the
> user is not at the pc, then some frames are duplicated on each computer. These can
easily be
> deleted.

And I will counter with the argument that it would be silly to modify the program
to allow backward frame processing and not also add the ability to detect what
the other instance is working on so that overlapping frames are not rendered.
I'm not sure how easy it would be to do in a cross platform environment.

-- 
Ken Tyler

Post a reply to this message

"Ken" <tyl### [at] pacbellnet> wrote in message
news:3DD50912.DBA86F4A@pacbell.net...
>
> And I will counter with the argument that it would be silly to modify the
program
> to allow backward frame processing and not also add the ability to detect what
> the other instance is working on so that overlapping frames are not rendered.
> I'm not sure how easy it would be to do in a cross platform environment.

Whilst that would be the ideal, in many practical cases, communication between
the two instances would not be possible in any event - e.g. sharing a render
between 2 non-networked pcs.

Post a reply to this message