Does anyone know if there is an effective maximum to the resolution you can
render at?

Background of question:
I am rendering some very high-object images as well as some high-detail ones at
times and would like to up the resolution significantly. I added a 12800x10240
resolution that has been beneficial but the ultimate goal would be a 41000x41000
image. However each time I have attempted to render that Pov-Ray has crashed on
me. Even when using a single, simple test object (a plain white sphere that
should use a single pixel). So I think this is running into a program limitation
at present.

In case anyone wonders about the max I have done so far, it is the 12800x10240
which is representing ~8,000,000 simple objects (plain white spheres). This has
rendered successfully (although parsing the CSV with all the locations takes a
long time and a TON of memory).

Computer rendering has:
AMD Phenom II Hexacore 2.8Ghz
8GB RAM
16GB Virtual Memory

Post a reply to this message

On 29.09.10 16:23, fillibar wrote:
> Does anyone know if there is an effective maximum to the resolution you can
> render at?

That depends...

> Computer rendering has:
> AMD Phenom II Hexacore 2.8Ghz
> 8GB RAM
> 16GB Virtual Memory

Nice, but I suppose you are running a 32 bit Windows XP on it? That means 
you have a maximum of 2 GB (default, or 3 GB with a XP boot option) per 
process of memory available. A 41000*41000 24 bit image already consumes 4.7 
GB of memory, which a 32 bit system cannot handle. You need to use a 64 bit 
Windows with the 64 bit version of POV-Ray.

	Thorsten

Post a reply to this message

Le 29/09/2010 16:23, fillibar nous fit lire :
> Does anyone know if there is an effective maximum to the resolution you can
> render at?
> 
> Background of question:
> I am rendering some very high-object images as well as some high-detail ones at
> times and would like to up the resolution significantly. I added a 12800x10240
> resolution that has been beneficial but the ultimate goal would be a 41000x41000
> image. However each time I have attempted to render that Pov-Ray has crashed on
> me. Even when using a single, simple test object (a plain white sphere that
> should use a single pixel). So I think this is running into a program limitation
> at present.

Sort of.
The issue seems to be in image/image.cpp, Image::Create (w=41000,
h=41000, t unknown, f=true)
Turn out that It's allocating RGBT_Float (t) as MemoryRGBFTImage which
does not even trigger the try/catch but goes SegFault directly.

An option FILE_MAPPED_IMAGE_ALLOCATOR seems to be possible, but is not
active by default (configure does not know about it), and in beta 38 at
least, the relevant code does not compile when CPPFLAGS is set to
-DFILE_MAPPED_IMAGE_ALLOCATOR .

> 8GB RAM
> 16GB Virtual Memory
> 
> 

A memory RGBFT_Float image of 41000 x 41000 is going to need at least 5
floats (single precision), that's a bit large for the poor default
implementation based on vector.
The iterator gets an issue: if it was a single memory block, direct
access, it would already reach the size of more than 31 Giga bytes
(assuming a float on 4 bytes)...

In the following code, replacing the (4) with (o) get a nice process
which use 31.3 GB;
But g++ has an issue if the code is "size_t o=41000*41000*5":
a.cpp: In function ‘int main()’:
a.cpp:9: warning: integer overflow in expression



using namespace std;
#include <stdio.h>
#include <vector>
typedef vector<float, allocator<float> > PixelContainer;

main()
{
PixelContainer a;
size_t o=41000;
o*=41000;
o*=5;
a.resize(4);
printf("vector: %zd\nSizeof %lu\n",a.max_size(),sizeof(size_t));
}


PS: arithmetic on unsigned int (as RGBFTImage creator is using) will
never get promoted to size_t by the compiler itself. But that does not
prevent an issue of overflow later: 41000² * 5 *4 = 0x00 07 D3 E8 7D 00
(beware 48 bits values, using already 36 bits), with "D3 E8 7D 00" being
a very huge number (negative).

In fact the resize parameter value is 0x01 F4 FA 1F 40
5 being signed, that's a signed value (well "F4FA1F40" is signed),
which lead probably with sign extension to requesting a resize to
0x FF FF FF FF F4 FA 1F 40
That's just too far for vector, as max_size is 0x3F FF FF FF FF FF FF FF

Or resizing to negative size just remove the vector, and there is no
iterator ? (difficult to see with -O3, and fastmath is just removing all
checks)


PS2: me running on linux64 with lot of memory( but not enough for 31.3
GB, at least I survive)

Post a reply to this message

I'm on something : a.resize(-1) in previous code (previous message) lead to:

terminate called after throwing an instance of 'std::length_error'
  what():  vector::_M_fill_insert
Abandon (core dumped)


Looks like vector does not like negative resize (due to promoting int(32
bits) to size_t after computation)

Post a reply to this message

On 30/09/2010 03:41, Le_Forgeron wrote:
> I'm on something : a.resize(-1) in previous code (previous message) lead to:
> 
> terminate called after throwing an instance of 'std::length_error'
>   what():  vector::_M_fill_insert
> Abandon (core dumped)
> 
> 
> Looks like vector does not like negative resize (due to promoting int(32
> bits) to size_t after computation)

Thanks for tracking this down - please feel free to raise a bug on it.

-- Chris

Post a reply to this message

Thorsten Froehlich <tho### [at] trfde> wrote:
> On 29.09.10 16:23, fillibar wrote:
> > Does anyone know if there is an effective maximum to the resolution you can
> > render at?
>
> That depends...
>
> > Computer rendering has:
> > AMD Phenom II Hexacore 2.8Ghz
> > 8GB RAM
> > 16GB Virtual Memory
>
> Nice, but I suppose you are running a 32 bit Windows XP on it? That means
> you have a maximum of 2 GB (default, or 3 GB with a XP boot option) per
> process of memory available. A 41000*41000 24 bit image already consumes 4.7
> GB of memory, which a 32 bit system cannot handle. You need to use a 64 bit
> Windows with the 64 bit version of POV-Ray.
>
>  Thorsten

I guess I should have mentioned I am running Windows 7 64bit. It would not have
made sense to buy that much memory on 32bit as you pointed out. I should also
mention I am running Beta 39 because earlier renders needed to take full
advantage of the 6 cores and the betas are the way to go.

Post a reply to this message

Le_Forgeron <jgr### [at] freefr> wrote:
> I'm on something : a.resize(-1) in previous code (previous message) lead to:
>
> terminate called after throwing an instance of 'std::length_error'
>   what():  vector::_M_fill_insert
> Abandon (core dumped)
>
>
> Looks like vector does not like negative resize (due to promoting int(32
> bits) to size_t after computation)

Wow! I did not expect someone to dig as deeply into this as you have, but thank
you very much. It looks like it is pretty clear at this point that the 41K x 41K
image is going to be beyond the capabilities for a little while at least. I will
just have to come up with a way to efficiently break the image into smaller
segments and recombine them although that is definitely sub-optimal.

Thanks for the information.

Post a reply to this message

Thorsten Froehlich <tho### [at] trfde> wrote:
> Nice, but I suppose you are running a 32 bit Windows XP on it? That means 
> you have a maximum of 2 GB (default, or 3 GB with a XP boot option) per 
> process of memory available. A 41000*41000 24 bit image already consumes 4.7 
> GB of memory, which a 32 bit system cannot handle. You need to use a 64 bit 
> Windows with the 64 bit version of POV-Ray.

  IIRC at least older versions of POV-Ray were designed such that, if you
were not using image preview, it would just stream the resulting image to
the output file rather than keeping it all in memory, and hence the only
limitation to image size is the maximum image size supported by the image
file format (and file system).

  Does the current version of POV-Ray require for the entire image to be
kept in memory?

-- 
                                                          - Warp

Post a reply to this message

On 09/29/2010 06:54 PM, Thorsten Froehlich wrote:

> Nice, but I suppose you are running a 32 bit Windows XP on it? That
> means you have a maximum of 2 GB (default, or 3 GB with a XP boot
> option) per process of memory available. A 41000*41000 24 bit image
> already consumes 4.7 GB of memory, which a 32 bit system cannot handle.

    Did you mean that POVray keep the full generated picture in memory
    when rendering it ?

Post a reply to this message

On 1/10/2010 02:25, tth wrote:
> Did you mean that POVray keep the full generated picture in memory
> when rendering it ?

Generally, yes. This is not optimal for huge renders, but for the types of
ones most users do it's an optimization that increases speed. Keep in mind
that as the SMP version of POV-Ray can render portions of the image out of
sequential order, it would complicate the image code quite a bit if we had
to be able to create the image in advance and then read and write any
arbitrary block (this is doubly so for files that use compression), and
would have a performance impact due to all the disk I/O.

We could cache the data to an intermediate file in raw format (and in fact
there is some code in there for this, however it's not currently working
properly), but while that bypasses the issue of the image file format, it
still leaves us with the I/O penalty.

Ideally we'd have an INI option to tell POV when to stop using RAM and
start using disk for the temporary image cache, however at this point there
is no setting for it.

-- Chris

Post a reply to this message