I'm trying to understand the communication between the backend and frontend,
particularly when the backend requests blocks of data.

I'd originally thought that block data was being passed from the frontend to the
backend, but I now think that only a serial number is passed and the block data
is found from elsewhere?

Any help on this would be very useful.

Post a reply to this message

"Jack Burton" <nomail@nomail> wrote:
> I'm trying to understand the communication between the backend and frontend,
> particularly when the backend requests blocks of data.

I've made some progress on understanding the communication between the frontend
and backend, but I still need some clarification (and some painkillers).

I'm attempting to make a distributed version of POV-Ray 3.7 using Open MPI. Most
of the issues are ensuring that the distributed POV-Ray instances (slaves) get
the next rectangle/block/tile from a master. I've been trying to isolate exactly
where to put in various Open MPI commands to send and receive data across nodes.
I originally looked around the GetNextRectangle() and StartRender() methods, but
I think I've stumbled onto a possible solution that'll fix distribution (but not
the image combination, that can come later).

Would be possible to globally share the various lists and mutexes from the
maste, and update them across a number of slaves that do the rendering, so when
each slaves backend rendering thread requests data, they do so from an
up-to-date list.

I hope that's not too vague a question. I'm still trying to understand POV-Ray's
structure. Feel free to correct me!

Thanks.

Post a reply to this message

On 30.06.11 17:12, Jack Burton wrote:
> "Jack Burton"<nomail@nomail>  wrote:
>> I'm trying to understand the communication between the backend and frontend,
>> particularly when the backend requests blocks of data.
>
> I've made some progress on understanding the communication between the frontend
> and backend, but I still need some clarification (and some painkillers).

LOL, that is (mostly) my code you are looking at ... so I guess I can answer 
your questions ;-)

> I'm attempting to make a distributed version of POV-Ray 3.7 using Open MPI. Most
> of the issues are ensuring that the distributed POV-Ray instances (slaves) get
> the next rectangle/block/tile from a master. I've been trying to isolate exactly
> where to put in various Open MPI commands to send and receive data across nodes.
> I originally looked around the GetNextRectangle() and StartRender() methods, but
> I think I've stumbled onto a possible solution that'll fix distribution (but not
> the image combination, that can come later).

It really depends on what you want to distribute. The GetNextRectangle 
method needs to be fast (and the lists that someone added there will go 
again in a future version as they are a major slowdown, but they will stay 
in 3.7 for now). GetNextRectangle is not the best candidate for a slow 
distributed function, unless you want your instances to mostly idle.

This sitting idle can be solved in several ways: The instances' speed all 
depends on smart assignment of the next block early on. You can select a 
fixed randomised block pattern as simplest solution. Or you assign blocks in 
batches. Or, smarter yet, you start the current block and then immediately 
request the next block (from a different thread), effectively hiding the 
latency of finding the next block.

Or, of course, you limit OpenMPI to the frontend only and run independent 
backends. This may (or may not, depending on how smart you make the 
frontend) mean you can most effectively render animations. The benefit here 
is that you need next to no changes to the core code, and also you don't 
need to bother with the rather difficult o distribute features, like radiosity.

Either way, be advised that there is some code in 3.7 RC that will also be 
in 3.7, but get cleaned up for 3.7.1 as currently the priority is not on 
easy to understand code, but on getting 3.7 released.

> Would be possible to globally share the various lists and mutexes from the
> master, and update them across a number of slaves that do the rendering, so when
> each slaves backend rendering thread requests data, they do so from an
> up-to-date list.

I am not sure why you would want to globally share mutexes, and I am not 
sure what lists you are referring to specifically, so i cannot really give 
you an answer here.

	Thorsten

Post a reply to this message

Hey,Thorsten.

Thanks for your reply!

> Or, of course, you limit OpenMPI to the frontend only and run independent
> backends. This may (or may not, depending on how smart you make the
> frontend) mean you can most effectively render animations. The benefit here
> is that you need next to no changes to the core code, and also you don't
> need to bother with the rather difficult o distribute features, like
> radiosity.

Yes, the idea is to modify the frontend only, but I need to understand
what/where the crucial data is that specifies which thread does what block. Open
MPI (as an API at least) isn't all that complex to use, it's just identifying
the correct place to make the changes! ;)

> I am not sure why you would want to globally share mutexes, and I am not
> sure what lists you are referring to specifically, so i cannot really give
> you an answer here.

I'm referring to the lists (e.g. blockBusyList) and mutexes (e.g.
nextBlockMutex) in view.h.

The idea of sharing the lists and mutexes is to record which blocks which have
been rendered and which blocks are being rendered, so all of the slaves in the
distributed setup can render the correct block and avoid duplication of blocks
and avoid unnecessary communication. The mutex (as I understand it) is to stop
the local threads accessing the lists at the same time, so this would be needed
for a distributed scenario, too; basically the lists need to be thread-safe
across multiple slaves.

During initialisation, I think each slave needs the parse the scene and have a
copy of the viewdata...but I'm still a little unclear on this. Anything you can
point to help me understand would be great.

This probably isn't the best design, but it probably is the fastest to code as I
need to wrap this up in the next month or so.

Post a reply to this message

On 01.07.11 12:24, Jack Burton wrote:
>> I am not sure why you would want to globally share mutexes, and I am not
>> sure what lists you are referring to specifically, so i cannot really give
>> you an answer here.
>
> I'm referring to the lists (e.g. blockBusyList) and mutexes (e.g.
> nextBlockMutex) in view.h.
>
> The idea of sharing the lists and mutexes is to record which blocks which have
> been rendered and which blocks are being rendered, so all of the slaves in the
> distributed setup can render the correct block and avoid duplication of blocks
> and avoid unnecessary communication. The mutex (as I understand it) is to stop
> the local threads accessing the lists at the same time, so this would be needed
> for a distributed scenario, too; basically the lists need to be thread-safe
> across multiple slaves.

Well, it depends on your radiosity needs, but as you cannot distribute the 
radiosity data anyway, you are restricted to recomputed radiosity samples. 
Thus, you can simply remove those lists and the associated code, they were 
only added to support radiosity sampling at render time in 3.7. Otherwise, 
they serve no purpose.

	Thorsten

Post a reply to this message

On 01.07.11 19:18, Thorsten Froehlich wrote:
> On 01.07.11 12:24, Jack Burton wrote:
>>> I am not sure why you would want to globally share mutexes, and I am not
>>> sure what lists you are referring to specifically, so i cannot really give
>>> you an answer here.
>>
>> I'm referring to the lists (e.g. blockBusyList) and mutexes (e.g.
>> nextBlockMutex) in view.h.
>>
>> The idea of sharing the lists and mutexes is to record which blocks which
>> have
>> been rendered and which blocks are being rendered, so all of the slaves in
>> the
>> distributed setup can render the correct block and avoid duplication of
>> blocks
>> and avoid unnecessary communication. The mutex (as I understand it) is to
>> stop
>> the local threads accessing the lists at the same time, so this would be
>> needed
>> for a distributed scenario, too; basically the lists need to be thread-safe
>> across multiple slaves.
>
> Well, it depends on your radiosity needs, but as you cannot distribute the
> radiosity data anyway, you are restricted to recomputed radiosity samples.
> Thus, you can simply remove those lists and the associated code, they were
> only added to support radiosity sampling at render time in 3.7. Otherwise,
> they serve no purpose.

This is the code really needed. Everything else is non-essential for your 
use-case (sorry for the broken formatting):

bool ViewData::GetNextRectangle(POVRect& rect, unsigned int& serial)
{
	Mutex::ScopedLock lock(nextBlockMutex);

	while(true)
	{
		if(nextBlock >= (blockWidth * blockHeight))
			return false;

		unsigned int blockX = nextBlock % blockWidth;
		unsigned int blockY = nextBlock / blockWidth;

		rect.left = renderArea.left + (blockX * blockSize);
		rect.right = min(renderArea.left + ((blockX + 1) * blockSize) - 1, 
renderArea.right);
		rect.top = renderArea.top + (blockY * blockSize);
		rect.bottom = min(renderArea.top + ((blockY + 1) * blockSize) - 1, 
renderArea.bottom);

		nextBlock++;
	}

	pixelsPending += rect.GetArea();

	serial = nextBlock;
	nextBlock++;

	return true;
}

As for the other code that was added, I am sorry, its not mine, and all I 
know it that it has several issues I will need to fix post-3.7.

As for your question about parsing, indeed you will need to do that in a 
distributed manner.

	Thorsten

Post a reply to this message

Am 01.07.2011 19:18, schrieb Thorsten Froehlich:
> On 01.07.11 12:24, Jack Burton wrote:
>>> I am not sure why you would want to globally share mutexes, and I am not
>>> sure what lists you are referring to specifically, so i cannot really
>>> give
>>> you an answer here.
>>
>> I'm referring to the lists (e.g. blockBusyList) and mutexes (e.g.
>> nextBlockMutex) in view.h.
>>
>> The idea of sharing the lists and mutexes is to record which blocks
>> which have
>> been rendered and which blocks are being rendered, so all of the
>> slaves in the
>> distributed setup can render the correct block and avoid duplication
>> of blocks
>> and avoid unnecessary communication. The mutex (as I understand it) is
>> to stop
>> the local threads accessing the lists at the same time, so this would
>> be needed
>> for a distributed scenario, too; basically the lists need to be
>> thread-safe
>> across multiple slaves.
>
> Well, it depends on your radiosity needs, but as you cannot distribute
> the radiosity data anyway, you are restricted to recomputed radiosity
> samples. Thus, you can simply remove those lists and the associated
> code, they were only added to support radiosity sampling at render time
> in 3.7. Otherwise, they serve no purpose.

To be precise, they were introduced to:

(a) provide an (optional) means to achieve full reproducibility of 
radiosity pretracing (otherwise thread scheduling will have a 
non-deterministic impact on the pretracing process and consequently on 
the final render result); and

(b) allow for adaptive radiosity pretrace, i.e. have a different number 
of pretrace steps for each render block depending on earlier pretrace 
step results

So even for radiosity renders those lists aren't strictly necessary (a 
bit of care must be taken though when removing them).

Post a reply to this message