POV-Ray : Newsgroups : povray.programming : Df3 file bigger than 1024^3 Server Time: 26 Sep 2020 09:23:58 GMT
  Df3 file bigger than 1024^3 (Message 1 to 10 of 10)  
From: arm2arm
Subject: Df3 file bigger than 1024^3
Date: 29 Aug 2006 09:55:00
Message: <web.44f40e4698b0d9e4b399450c0@news.povray.org>
Hello,
I tried to render df3 volume file with 2048x2048x512 data and
povray crushes. the reason was pointer index for map.It was out from integer
range in the df3 reading routine in the file:
pattern.cpp
/////////////
line 4288: void Read_Density_File(DENSITY_FILE *df)
{
  // int x, y, z, sx, sy, sz;
 long x, y, z, sx, sy, sz;//changed int to long
///////////////

This works for reading of df3 file, but after parsing it is crushing
/////////
0:00:00 Rendering line 1 of 240Segmentation fault
/////////////////////

Where I need to change (x, y, z, sx, sy, sz) to get the run for huge data?
Thanks

Arman.

+++++++++++++++++++++++++++++++++
Arman Khalatyan
Address:
Astrophysical Institute Potsdam,
An der Sternwarte 16,
D-14482 Potsdam, GERMANY
Phones: (+049) 0331 7499517(work)


Post a reply to this message

From: Warp
Subject: Re: Df3 file bigger than 1024^3
Date: 29 Aug 2006 11:16:06
Message: <44f421f6@news.povray.org>
arm2arm <nomail@nomail> wrote:
> I tried to render df3 volume file with 2048x2048x512 data

  That amount exactly hits the 2GB barrier. Are you using the 64-bit
version of POV-Ray (which naturally requires you to have a 64-bit
system) or the 32-bit version?

  If you are using the 64-bit version of POV-Ray then it is very possible
that there may still be some non-64-bit-compatible code left in POV-Ray
regarding this. However, it's important to know your system specs before
any further investigation.

> and povray crushes.

  Btw, it's "crashes".

-- 
                                                          - Warp


Post a reply to this message

From: arm2arm
Subject: Re: Df3 file bigger than 1024^3
Date: 29 Aug 2006 11:55:01
Message: <web.44f429312dd4e7f6b399450c0@news.povray.org>
Sorry, the system is:
64bit
POV-Ray 3.6.1 compiled with PGI compiler and g++ 4.0.2 20050901 (Both are
segmenting )
RAM 32G
4CPU SMP maschine with  AMD Opteron(tm) Processors 850.
If array size is 2048*2048*512 it is little bit bigger than Max_INT 2^31-1.
Therefore when we trying to read with int index in the file pattern.cpp the
index for array offset is calculated wrong.
also in the line when we try to check file type of df3 it is always telling
wrong file type due to the line/s: 4315
 // figure out the data size
  if((sx * sy *sz*4) == len)// this is always false if file size is bigger
4G


arman.


+++++++++++++++++++++++++++++++++
Arman Khalatyan
Address:
Astrophysical Institute Potsdam,
An der Sternwarte 16,
D-14482 Potsdam, GERMANY
Phones: (+049) 0331 7499517(work)


Post a reply to this message

From: Warp
Subject: Re: Df3 file bigger than 1024^3
Date: 29 Aug 2006 15:12:18
Message: <44f45952@news.povray.org>
arm2arm <nomail@nomail> wrote:
> Sorry, the system is:
> 64bit

  I notified this issue to the povteam. It may indeed be that that part
of the code has not been fixed for 64-bit compatibility.

-- 
                                                          - Warp


Post a reply to this message

From: Christoph Hormann
Subject: Re: Df3 file bigger than 1024^3
Date: 29 Aug 2006 16:45:03
Message: <ed1qmb$j1i$1@chho.imagico.de>
arm2arm wrote:
> Hello,
> I tried to render df3 volume file with 2048x2048x512 data and
> povray crushes. the reason was pointer index for map.It was out from integer
> range in the df3 reading routine in the file:
> pattern.cpp
> /////////////
> line 4288: void Read_Density_File(DENSITY_FILE *df)
> {
>   // int x, y, z, sx, sy, sz;
>  long x, y, z, sx, sy, sz;//changed int to long
> ///////////////
> 
> This works for reading of df3 file, but after parsing it is crushing
> /////////
> 0:00:00 Rendering line 1 of 240Segmentation fault
> /////////////////////
> 
> Where I need to change (x, y, z, sx, sy, sz) to get the run for huge data?
> Thanks

The density file pattern is apparently indeed currently not suited for 
being used above the 2GB limit.  This is the case both in the DF3 
reading you pointed out and the actual pattern evaluation.  The required 
changes are probably not very difficult but only recommended to make for 
someone who knows the POV-Ray source code.

Testing this kind of thing is a bit cumbersome of course - having 2GB+ 
df3s lying around and reading them...

Christoph

-- 
POV-Ray tutorials, include files, Landscape of the week:
http://www.imagico.de/ (Last updated 20 Aug. 2006)
MegaPOV with mechanics simulation: http://megapov.inetart.net/


Post a reply to this message

From: Christoph Hormann
Subject: Re: Df3 file bigger than 1024^3
Date: 29 Aug 2006 18:45:03
Message: <ed21u8$q8t$1@chho.imagico.de>
I made a patch for MegaPOV probably solving it (for 64bit systems only 
of course):

http://megapov.imagico.de/cgi-bin/hgweb.cgi?cs=649a35101349

Note i tested it only with the standard 'galaxy.pov' - not with a large 
file.  Please check if it fixes your problem (the changes can also 
easily be integrated into plain POV-Ray 3.6.1 manually if you don't want 
to install Megapov).

Christoph

-- 
POV-Ray tutorials, include files, Landscape of the week:
http://www.imagico.de/ (Last updated 20 Aug. 2006)
MegaPOV with mechanics simulation: http://megapov.inetart.net/


Post a reply to this message

From: arm2arm
Subject: Re: Df3 file bigger than 1024^3
Date: 30 Aug 2006 15:55:00
Message: <web.44f5b42d2dd4e7f6b399450c0@news.povray.org>
Hi Christoph,
thank you for patch,
I never used megapov, can you provide the link for source code of megapov?
Thanks.
Arman.


Post a reply to this message

From: Nicolas Calimet
Subject: Re: Df3 file bigger than 1024^3
Date: 30 Aug 2006 16:42:02
Message: <44f5bfda@news.povray.org>
> I never used megapov, can you provide the link for source code of megapov?

http://megapov.inetart.net/download.html


	Here are some instructions to apply the patch to the megapov for Unix
source code:

1) Download the "generic Unix source version", for instance:
       http://megapov.inetart.net/packages/unix/megapov-1.2.1.tar.bz2

2) Unpack it (creates the megapov-1.2.1/ directory):
       tar xvjf megapov-1.2.1.tar.bz2

3) View Christoph's patch in raw form, and save it as a text file in your
    home directory, e.g. ~/mp.patch:
       http://megapov.imagico.de/cgi-bin/hgweb.cgi?cs=649a35101349;style=raw

4) Go to the megapov-1.2.1/ source directory and apply the patch:
       cd megapov-1.2.1/
       patch -p1 < ~/mp.patch

    You should get something like this:
patching file source/frame.h
patching file source/patches/patches.h
Hunk #1 succeeded at 187 with fuzz 2 (offset -7 lines).
patching file source/pattern.cpp
Hunk #1 succeeded at 1739 (offset -279 lines).
Hunk #2 succeeded at 1761 (offset -279 lines).
Hunk #3 succeeded at 1780 (offset -279 lines).
Hunk #4 succeeded at 1807 (offset -279 lines).
Hunk #5 succeeded at 1871 (offset -279 lines).
Hunk #6 succeeded at 4607 (offset -332 lines).
Hunk #7 succeeded at 4634 (offset -332 lines).
Hunk #8 succeeded at 4651 (offset -332 lines).
Hunk #9 succeeded at 4668 (offset -332 lines).

5) Run the usual  ./configure COMPILED_BY=your_name  &&  make

	- NC


Post a reply to this message

From: arm2arm
Subject: Re: Df3 file bigger than 1024^3
Date: 30 Aug 2006 17:05:01
Message: <web.44f5c4822dd4e7f6b399450c0@news.povray.org>
COOL!
Everything is working for me,
I rendered 8Gb data.
Only the bottleneck is reading of df3 file,
I changed the zyx loops to z loop over xy slabs.
Thank you again.

Arman.


Post a reply to this message

From: Raf256
Subject: Re: Df3 file bigger than 1024^3
Date: 4 Sep 2006 20:24:44
Message: <44fc8b8c@news.povray.org>
arm2arm <web.44f5c4822dd4e7f6b399450c0@news.povray.org> Wednesday 30 of
August 2006 19:01

> COOL!
> Everything is working for me,
> I rendered 8Gb data.
> Only the bottleneck is reading of df3 file,
> I changed the zyx loops to z loop over xy slabs.
> Thank you again.

Interesting, can You share the patch that does it?

-- 
Rafa³ Maj
  > wiesz. Ja wiem co to kernel. I wiem sporo o linuksie.
  > Nawet dzia³a³em przez jakie¶ 4 godziny na linuksie - knoppixie.
  (C) by DeJotPe - niezale¿y ekspert i wytrawny krytyk Linuksa.


Post a reply to this message

Copyright 2003-2008 Persistence of Vision Raytracer Pty. Ltd.