POV-Ray : Newsgroups : povray.unix : Bug in 3.7.0unofficial ? Server Time: 23 Oct 2019 02:20:59 GMT
  Bug in 3.7.0unofficial ? (Message 1 to 5 of 5)  
From: Jörg "Yadgar" Bleimann
Subject: Bug in 3.7.0unofficial ?
Date: 26 Sep 2019 00:16:51
Message: <5d8c0373$1@news.povray.org>
Hi(gh)!

Recently, I started generation "color profiles" from ripped DVDs, using 
the following script:

#declare 
path="/home/Video/Filme/Filme_in_Bearbeitung/movie/Durchschnittsfarbpixel/"
#declare file="movie_1pixel-"
#declare f=1;
#declare fac=25;

#declare F_Incandescent=
finish { ambient 1 diffuse 0 }

#declare T_White=
texture
{
   pigment { color rgb 1 }
   finish { F_Incandescent }
}

#declare frames = 124819;
#if (frames/fac-div(frames, fac) < 0.5)
   #declare striplength=floor(frames/fac);
#else
   #declare striplength=ceil(frames/fac);
#end

#declare image_w=striplength+100;
#declare image_center=image_w/2;
#declare strip_offset=<50, 60, 0>;

#declare StripBorderLong=
box
{

   <0, 0, -0.01>, <striplength+4, 2, 1>
   texture { T_White }
}

#declare StripBorderShort=
box
{
   <0, 0, 0>, <2, 20, 1>
   texture { T_White }
}

#declare Minute=
box
{
   0, <2, 3, 1>
   texture { T_White }
}

#declare FiveMinutes=
box
{
   0, <2, 5, 1>
   texture { T_White }
}


#include "functions.inc"

camera
{
   orthographic
   right <image_w, 0, 0>
   up <0, 100, 0>
   location <image_center, 50, -10>
   look_at <image_center, 50, 0>
}

/* background
{
   color rgb 1
} */


#while (f < frames)
   #declare g=0;
   #declare sumcolvect=<0, 0, 0>;
   #while (g < fac & f+g <= frames)
     #declare P_Pixel=
     pigment
     {
       image_map
       {
	#declare filefull =  concat(path, file, str(f+g, -6, 0), ".png");
	png filefull
       }
     }
     #declare sumcolvect=sumcolvect+eval_pigment(P_Pixel, <0.5, 0.5, 0>);
     #declare g=g+1;
   #end
   #declare C_Avgcolor=sumcolvect/fac;
   #declare AvgcolorPixel=
   box
   {
     0, <1, 20, 1>
     texture
     {
       pigment { color rgb C_Avgcolor }
       finish { F_Incandescent }
     }
   }
   object { AvgcolorPixel translate strip_offset+<div(f, 25), -20, 0> }
   #declare f=f+fac;
#end

object { StripBorderLong translate strip_offset-<2, 0, 0> }
object { StripBorderLong translate strip_offset-<2, 22, 0> }
object { StripBorderShort translate strip_offset-<2, 20, 0> }
object { StripBorderShort translate strip_offset+<striplength, -20, 0> }

#declare minspace=1500/fac;

#declare p=0;
#while (p*minspace < striplength)
   object
   {
     #if (mod(p, 5) = 0)
       FiveMinutes
     #else
       Minute
     #end
     translate strip_offset+<p*minspace-1, 2, 0>
   }
   #if (mod(p, 5) = 0)
     #declare hours=div(p, 60);
     #declare minutes = p-hours*60;
     text
     {
       ttf "../Fonts/arial.ttf"
       concat(str(hours, 1, 0), ":", str(minutes, -2, 0), "'")
       1, 0
       scale 20
       translate strip_offset+<p*minspace-20, 12, 0>
       texture { T_White }
     }
   #end
   #declare p=p+1;
#end

It worked a few times, but after this, I only get various error 
messages, the last one was:

povray: malloc.c:2372: sysmalloc: Assertion `(old_top == (((mbinptr) 
(((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct 
malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= 
(unsigned long)((((__builtin_offsetof (struct malloc_chunk, 
fd_nextsize))+((2 *(sizeof(size_t))) - 1)) & ~((2 *(sizeof(size_t))) - 
1))) && ((old_top)->size & 0x1) && ((unsigned long) old_end & pagemask) 
== 0)' failed.

Or is my RAM about to disintegrate?

See you in Khyberspace!

Yadgar


Post a reply to this message

From: William F Pokorny
Subject: Re: Bug in 3.7.0unofficial ?
Date: 26 Sep 2019 14:08:32
Message: <5d8cc660$1@news.povray.org>
On 9/25/19 8:18 PM, Jörg "Yadgar" Bleimann wrote:
> Hi(gh)!
> 
> Recently, I started generation "color profiles" from ripped DVDs, using 
> the following script:
> 
...
> 
> Or is my RAM about to disintegrate?
> 
> See you in Khyberspace!
> 
> Yadgar

I suspect your running out of memory though looking at your code not 
jumping at me out why. Maybe a memory leak unless someone picks up 
something I missed. Looks like you are loading a large number of single 
pixel images? I'd hope these freed on redef, but maybe they aren't or 
something. Don't know.

Are you able to run a linux command like top in an xterm - or a fancier 
system monitor as your job runs? A command line like time (on Ubuntu at
/usr/bin/time) which can be used like:

/usr/bin/time povray the.pov

or

/usr/bing/time --verbose -- povray the.pov

which kick out a max resident memory value in the former and more 
information in the later case. Are you using a large amount of memory 
even when your script completes OK?

There have been memory related updates/fixes to v3.8 master if your able 
to try that.

If you are able to run it sometimes, does it run when you have fewer 
frames or some other specific conditions. Might give a clue.

If not able to isolate better yourself a test script/scene which uses 
maybe the same image over and over but still runs out of memory would be 
useful in debugging I guess.

Bill P.


Post a reply to this message

From: Jörg "Yadgar" Bleimann
Subject: Re: Bug in 3.7.0unofficial ?
Date: 26 Sep 2019 16:13:00
Message: <5d8ce38c$1@news.povray.org>
Hi(gh)!

On 26.09.19 16:08, William F Pokorny wrote:
 > I suspect your running out of memory though looking at your code not
 > jumping at me out why. Maybe a memory leak unless someone picks up
 > something I missed. Looks like you are loading a large number of single
 > pixel images? I'd hope these freed on redef, but maybe they aren't or
 > something. Don't know.

Strangely enough, it worked fine with POV-Ray 3.7 under Windows 7 on the 
same machine... so it's probably more likely a software (i. e. POV-Ray 
3.7.0unofficial for Linux) than a hardware issue!

Yes, I load between 125,000 and 300,000 one-pixel PNGs, each of them not 
larger than 90 bytes... what is "freed on redef"? Explain!


 > Are you able to run a linux command like top in an xterm - or a fancier
 > system monitor as your job runs?

Yes, that works!

  A command line like time (on Ubuntu at
 > /usr/bin/time) which can be used like:
I'm not using Ubuntu, but rather Debian 8.11 (oldoldstable)!


 > If you are able to run it sometimes, does it run when you have fewer
 > frames or some other specific conditions. Might give a clue.

I did not try this yet... but I will soon as I also want to process some 
shorter videos!
See you in Khyberspace!

Yadgar


Post a reply to this message

From: Jörg "Yadgar" Bleimann
Subject: Re: Bug in 3.7.0unofficial ?
Date: 26 Sep 2019 18:35:52
Message: <5d8d0508$1@news.povray.org>
Hi(gh)!

On 26.09.19 16:08, William F Pokorny wrote:
> I'd hope these freed on redef, but maybe they aren't or 
> something. Don't know.

Meanwhile, it dawned on me what you meant, so I added an #undef 
statement into the innermost loop:

   #while (g < fac & f+g <= frames)
     #declare P_Pixel=
     pigment
     {
       image_map
       {
	#declare filefull =  concat(path, file, str(f+g, -6, 0), ".png");
	png filefull
       }
     }
     #declare sumcolvect=sumcolvect+eval_pigment(P_Pixel, <0.5, 0.5, 0>);
     #undef P_Pixel
     #declare g=g+1;
   #end

...but all I get now is just another error message:

*** Error in `povray': free(): invalid size: 0x00007f91d007bc30 ***

See you in Khyberspace!

Yadgar


Post a reply to this message

From: William F Pokorny
Subject: Re: Bug in 3.7.0unofficial ?
Date: 26 Sep 2019 19:07:53
Message: <5d8d0c89$1@news.povray.org>
On 9/26/19 12:13 PM, Jörg "Yadgar" Bleimann wrote:
> Hi(gh)!
> 
> Strangely enough, it worked fine with POV-Ray 3.7 under Windows 7 on the 
> same machine... so it's probably more likely a software (i. e. POV-Ray 
> 3.7.0unofficial for Linux) than a hardware issue!
> 

This sounds like our old thread stack size issue where jobs would run on 
windows with a given allocation size and not on unix builds. Not sure we 
ever nailed down whether the windows compile was handling the stack 
overruns (by allocating more memory as needed) or if unix compiles 
needed more memory for some reason. As a patch, there was an initial 
macro implementation in v38 where one could set a larger allocation 
during the POV-Ray compile. Christoph later replaced it with a method 
based upon thread local storage and std::vector which, while a tiny bit 
slower, fixed all the storage overruns we'd seen. More memory is 
allocated if need be until you run out it instead of there being a fixed 
compile time set limit of memory for each thread's stack...

Long winded way to say using the current v38 branch might well fix your 
problem.

> Yes, I load between 125,000 and 300,000 one-pixel PNGs, each of them not 
> larger than 90 bytes... what is "freed on redef"? Explain!
> 

The slang was just me thinking aloud. When you declare again each pixel 
pigment { image_map .... } with the same ID/Name, it should be the 
memory used for the previous image_map is freed.

> 
>  > Are you able to run a linux command like top in an xterm - or a fancier
>  > system monitor as your job runs?
> 
> Yes, that works!
> 
>   A command line like time (on Ubuntu at
>  > /usr/bin/time) which can be used like:
> I'm not using Ubuntu, but rather Debian 8.11 (oldoldstable)!
> 

Ubuntu is based upon Debian so you probably have the /usr/bin.time 
command too. Note. If you are in a bash shell/xterm the shell itself has 
a 'time' command which isn't the same as the fully specified 
/usr/bin/time command. Using \time is another way to use /usr/bin/time 
over bash's time which should work. Yep, messy.

Being back on Debian 8.11 might mean you cannot compile the current v38 
master branch. I don't know what gnu/clang compiler versions it supports.

> 
>  > If you are able to run it sometimes, does it run when you have fewer
>  > frames or some other specific conditions. Might give a clue.
> 
> I did not try this yet... but I will soon as I also want to process some 
> shorter videos!
> See you in Khyberspace!
> 
> Yadgar

Let us know what you find.

Bill P.


Post a reply to this message

Copyright 2003-2008 Persistence of Vision Raytracer Pty. Ltd.