POV-Ray : Newsgroups : povray.advanced-users : Rendering extremely large images Server Time
16 May 2024 01:46:18 EDT (-0400)
  Rendering extremely large images (Message 1 to 9 of 9)  
From: Atlaste
Subject: Rendering extremely large images
Date: 30 Aug 2013 03:45:01
Message: <web.52204d6b7c8d2191ebcb0460@news.povray.org>
Hi,

For one of my hobby projects I've been attempting to build a very large map of
an island. Nothing fancy, just a height field with trees and grass on it and a
bunch of textures.

My intent is to make a very large render of this map and use it in a flash
application. The application will only show a small portion of the map at any
given time and the user can click around on it.

In an attempt to create this map I added a very large resolution to the
quickres.ini - 38400x21600 . So far that approach has only resulted in a totally
unresponsive pov-ray - and I think I even want a bigger resolution :-)

To solve this, I would like to stitch smaller images together.

So, what I've tried is to make an orthographic camera:

camera
{
    orthographic
    location <5,15,0>*0.05 right <1,0,0> up <0,1,0> look_at 0
}

At this point I figured that rendering multiple images can be done by sliding
the z-direction, say (for 10 images):

camera
{
    orthographic
    location <5,15,clock*9.8>*0.05 right <1,0,0> up <0,1,0> look_at
<0,0,clock*9.8>
}

(I've intentionally chosen 9.8 instead of 10 to have a small overlap that I can
use to stitch things together with standard software -- I haven't tested this
yet, but I suppose it should work)

However, that's about as far as I got; at this point I have no clue how to do
the same trick in the x direction.

Any help is appreciated!

Kind regards,
Stefan.


Post a reply to this message

From: Le Forgeron
Subject: Re: Rendering extremely large images
Date: 30 Aug 2013 07:42:18
Message: <5220851a@news.povray.org>
Le 30/08/2013 09:44, Atlaste a écrit :

> However, that's about as far as I got; at this point I have no clue how to do
> the same trick in the x direction.
> 
> Any help is appreciated!

How do you count the cells of a 10x10 table ?
you can start at the first row (0), and count from 0 to 9 on the column.
Then start at the second row with 10 and count up to 19.
And so on until 99.

So were is cell 38 ?
38 modulo 10 gives 8.
38 divided by 10 gives 3.
So cell 38 is in the fourth row, nineth column (assuming first row, firs
column is 00, an offset of +1 is needed for usual location)

your formula becomes something like:

#declare ColumnSize = 10;// or whatever the number of image per column
camera
{
    orthographic
    location <div(clock,ColumnSize)*9.8 ,
     15,
     mod(clock, ColumnSize)*9.8>

right <10,0,0>
direction <0,1,0>
up <0,0,10>
look_at <div(clock,ColumnSize)*9.8,0,mod(clock, ColumnSize)*9.8>
}


clock goes from 0 to (NumberOfRow)*(ColumnSize) by step of 1.

-- 
Just because nobody complains does not mean all parachutes are perfect.


Post a reply to this message

From: Atlaste
Subject: Re: Rendering extremely large images
Date: 30 Aug 2013 08:40:00
Message: <web.52209180a03553e0ebcb0460@news.povray.org>
> your formula becomes something like:
>
> #declare ColumnSize = 10;// or whatever the number of image per column
> camera
> {
>     orthographic
>     location <div(clock,ColumnSize)*9.8 ,
>      15,
>      mod(clock, ColumnSize)*9.8>
>
> right <10,0,0>
> direction <0,1,0>
> up <0,0,10>
> look_at <div(clock,ColumnSize)*9.8,0,mod(clock, ColumnSize)*9.8>
> }
>
>
> clock goes from 0 to (NumberOfRow)*(ColumnSize) by step of 1.

Yes, I've gotten that far; I just thought this was incorrect: the X axis will
work like that - sure, but because the Z axis is under an angle, shouldn't I
first calculate the plane orthogonal to the vector between the eye and the
look_at point and move on that? Or doesn't that matter because I use
orthographic projection?

Stefan.


Post a reply to this message

From: Bald Eagle
Subject: Re: Rendering extremely large images
Date: 30 Aug 2013 10:05:01
Message: <web.5220a661a03553e0dd2ebc560@news.povray.org>
Not sure if this will help any:
http://paulbourke.net/miscellaneous/povexamples/

Near the end is some info about making Quicktime navigable objects.

Maybe if you just run an animation and use photo stitching software on the
resulting string of renders, that will be faster?


Post a reply to this message

From: clipka
Subject: Re: Rendering extremely large images
Date: 30 Aug 2013 10:51:07
Message: <5220b15b$1@news.povray.org>
Am 30.08.2013 09:44, schrieb Atlaste:
> Hi,
>
> For one of my hobby projects I've been attempting to build a very large map of
> an island. Nothing fancy, just a height field with trees and grass on it and a
> bunch of textures.
>
> My intent is to make a very large render of this map and use it in a flash
> application. The application will only show a small portion of the map at any
> given time and the user can click around on it.
>
> In an attempt to create this map I added a very large resolution to the
> quickres.ini - 38400x21600 . So far that approach has only resulted in a totally
> unresponsive pov-ray - and I think I even want a bigger resolution :-)

That total unresponsiveness is perfectly normal when starting a render 
of an extremely large output image; POV-Ray needs to create a multi-GB 
temporary file, and Windows insists on initializing this file right from 
the start (rather than initializing only the blocks actually written to, 
as Linux does), which may take several minutes.

Rendering (including creation of the file) is probably still faster if 
you do it in one large render.


There's one other noteworthy caveat with big files: If the scene is 
extremely simple, memory consumption may rise during render as the 
render threads create image data faster than it can be stored in the 
temp file; this can ultimately lead to swapping. (Nothing too serious, 
fortunately; the system will remain responsive enough to handle the 
situation by pausing the render for a while; this will allow image data 
to be tucked away and memory be freed again).


Post a reply to this message

From: Warp
Subject: Re: Rendering extremely large images
Date: 30 Aug 2013 12:45:45
Message: <5220cc38@news.povray.org>
clipka <ano### [at] anonymousorg> wrote:
> That total unresponsiveness is perfectly normal when starting a render 
> of an extremely large output image; POV-Ray needs to create a multi-GB 
> temporary file, and Windows insists on initializing this file right from 
> the start (rather than initializing only the blocks actually written to, 
> as Linux does), which may take several minutes.

In a sense POV-Ray has gone slightly backwards when talking about
rendering extremely large images.

Previous versions of POV-Ray had the idea that they would keep in memory
just a couple of rendered pixel rows (and AFAIK this only if you use
antialiasing), flushing the rendered pixels directly to the output image
file as soon as they are rendered. If you had displaying turned off, this
meant that you could render an almost arbitrarily large image (only
limited by the output image file format and memory addressing limitations)
without any problems because only a couple of pixel rows were kept in
memory at most at any given time. Rendering something like a 65536x65536
image shouldn't be a problem with an older version of POV-Ray.

In other words, the idea was like "render-and-forget" (for each pixel row.)
This was quite efficient for rendering very large images.

POV-Ray 3.7, however, has to keep all kinds of temporary files and
buffers for the whole image. You will quickly run out of memory and/or
disk space, and have all kinds of inefficiencies, when rendering very
large images.

-- 
                                                          - Warp


Post a reply to this message

From: Atlaste
Subject: Re: Rendering extremely large images
Date: 1 Sep 2013 09:25:01
Message: <web.52233fdba03553e0ebcb0460@news.povray.org>
clipka <ano### [at] anonymousorg> wrote:
> That total unresponsiveness is perfectly normal when starting a render
> of an extremely large output image; POV-Ray needs to create a multi-GB
> temporary file, and Windows insists on initializing this file right from
> the start (rather than initializing only the blocks actually written to,
> as Linux does), which may take several minutes.

Ah thanks for the heads-up! So that's the problem... Yes I'm on Windows, and
I've been waiting for minutes and minutes... guess I haven't waited long enough.

I'm just wondering... what is large and how can I influence where that file is
saved? Say, if it's only a few GB, I can just as easily create a ramdisk (on the
same or another server) and store it there to speed things up? Or should I think
more in the line of half a TB?

> Rendering (including creation of the file) is probably still faster if
> you do it in one large render.

OK, it does make everything much easier, so if that's possible that would
definitely be my preferred approach...

> There's one other noteworthy caveat with big files: If the scene is
> extremely simple, memory consumption may rise during render as the
> render threads create image data faster than it can be stored in the
> temp file; this can ultimately lead to swapping. (Nothing too serious,
> fortunately; the system will remain responsive enough to handle the
> situation by pausing the render for a while; this will allow image data
> to be tucked away and memory be freed again).

Well, it started off simple... now there's 50k trees, about 200k grass objects,
water, rivers, clouds... so no, I guess it's not an extremely simple scene
anymore :-)

Thanks,
Stefan.


Post a reply to this message

From: Le Forgeron
Subject: Re: Rendering extremely large images
Date: 1 Sep 2013 10:40:00
Message: <522351c0$1@news.povray.org>
Le 01/09/2013 15:23, Atlaste nous fit lire :
> I'm just wondering... what is large and how can I influence where that file is
> saved? Say, if it's only a few GB, I can just as easily create a ramdisk (on the
> same or another server) and store it there to speed things up? Or should I think
> more in the line of half a TB?

Large (the criteria to use a temporary file instead of 100% ram) is
defined via the Max_Image_Buffer_Memory (+MI) parameter (default to 128,
number of megabytes, uncompressed floating points of 5 values, above
which a file is created)

So, for 38400x21600, you can force to stay in memory with either:
MI=0
or any MI > 15820

Beware, you need more than 16 Gigabytes of memory to avoid the
swap-deadland.

The file, when it exist, depends on the system:
* windows: GetTempPath(), extended with a povwin subdirectory if
possible (else raw value of GetTempPath() )
** GetTempPath checks in order : environment variables TMP, TEMP,
USERPROFILE, and the windows directory.


Post a reply to this message

From: Atlaste
Subject: Re: Rendering extremely large images
Date: 1 Sep 2013 17:05:01
Message: <web.5223abe4a03553e0ebcb0460@news.povray.org>
> Large (the criteria to use a temporary file instead of 100% ram) is
> defined via the Max_Image_Buffer_Memory (+MI) parameter (default to 128,
> number of megabytes, uncompressed floating points of 5 values, above
> which a file is created)
>
> So, for 38400x21600, you can force to stay in memory with either:
> MI=0
> or any MI > 15820
>
> Beware, you need more than 16 Gigabytes of memory to avoid the
> swap-deadland.
>
> The file, when it exist, depends on the system:
> * windows: GetTempPath(), extended with a povwin subdirectory if
> possible (else raw value of GetTempPath() )
> ** GetTempPath checks in order : environment variables TMP, TEMP,
> USERPROFILE, and the windows directory.

Ah, this is great info, exactly what I was looking for! 16 GB is not much of a
problem -- and even if it was, now I can simply do the math and work it out.

Also the info about the temp file is quite useful. My server has a solid state
disk, so if I can use that for all pov-ray activities, it'll probably save a lot
of time.

I'll give it a try, many thanks for all the useful info!

Stefan.


Post a reply to this message

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.