POV-Ray : Newsgroups : povray.general : memory leak in 3.1g? Server Time
11 Aug 2024 11:18:24 EDT (-0400)
  memory leak in 3.1g? (Message 7 to 16 of 16)  
<<< Previous 6 Messages Goto Initial 10 Messages
From: Steve
Subject: Re: memory leak in 3.1g?
Date: 21 Aug 1999 13:39:34
Message: <37BED4DF.6FF2C026@ndirect.co.uk>
So basicly I've got to get a hell of a lot more memory before
rendering that scene which takes 8 days to parse, the annoying
thing is that it only takes two and a half hours to parse the
first half, I know this because of putting #dbugs in to monitor
the parsing.  


Thorsten Froehlich wrote:
> 
> In article <37BDD20B.AAF0364A@ndirect.co.uk> , Steve <sjl### [at] ndirectcouk>
> wrote:
> > When parsing a scene that needs to swap for a few days it sounds
> > asthough the machine is working something out, placing an answer
> > to that in the swap file, reading that answer in order to find
> > out where an object goes or something, deleting that answer, and
> > putting the object place info inot the swap file and then
> > starting over again for the next object.
> 
> I think you misunderstand what a swap file does: When physical memory gets
> low the operating system (not POV-Ray!) will store some blocks of memory on
> a mass storage device (usually harddisk) and adjust the memory map of the
> CPU so the larger memory still appears to be there even for the processor.
> When it tries to access memory that is on disk, an exception handler (= a
> function) of the OS will be called and it will move some other blocks of
> memory to disk and move those needed from disk to memory. This processes is
> completely transparent to POV-Ray as it is to any other application.
> 
> > Would it be possible to have POV use memory until it's full and
> > then instead of starting to swap, putting say the last ten
> > percent of memory into the swap file and there fore having ten
> > percent of memory to play with unitl it's full and doing the same
> > again over and over rather than constantly reading and writing to
> > the swap file.
> 
> As said before, the swap file is managed by the operating system. Most
> operating system memory managers can determine efficiently which blocks of
> memory to move to disk and which are needed often and therefore should stay
> in memory.
> 
> > I don't know if this would be possible or not, but it would speed
> > those types of render up considerably.
> 
> While a specialized memory manager could surely squeeze out a few percent,
> it would basically require to write a whole operating system as memory
> management is obviously one of the core functions of any OS!
> 
>     Thorsten
> 
> ____________________________________________________
> Thorsten Froehlich, Duisburg, Germany
> e-mail: tho### [at] trfde
> 
> Visit POV-Ray on the web: http://mac.povray.org

-- 
Cheers
Steve

email - mailto:sjl### [at] ndirectcouk

%HAV-A-NICEDAY Error not enough coffee. 

web:  http://www.ndirect.co.uk/~sjlen/


Post a reply to this message

From: Charles
Subject: Re: memory leak in 3.1g?
Date: 21 Aug 1999 17:07:09
Message: <37BF14E8.9D17D1D3@enter.net>
Thorsten Froehlich wrote:
> As said before, the swap file is managed by the operating system. Most
> operating system memory managers can determine efficiently which blocks of
> memory to move to disk and which are needed often and therefore should stay
> in memory.

Aye, and a pity 'tis, too, considering a weird thing that just came
to light which I'm assuming (simply on general principals) is a
MS Windows thang...

I have a trace currently that exceeds my available memory considerably
during parse (from what I can see parsing seems to pull two to three
times the memory needed for the actual render itself, although it
releases the excess when finished). So anyway, it parses for a bit,
then starts to swap, then finally finishes parsing and returns a 
solid 30 megs of RAM, but get this... apparently the portion doled
to swapfile is still out there on the disk for some reason. This
conclusion I draw from the facts that A) The allocated plus free
mem exceeds what I have installed, and B) The harddrive thrashes
halfway to Montana and back when the trace finishes. 

You would think the OS is your friend. That it would say: Hey, I've
got 17 megs in swapfile, and 30 megs of physical RAM free; Wouldn't
my pal, the user, be pleased if I stuck this virtual mem back where
it belongs and boosted his performance? But alas, this doesn't
look like the case. 

What brings me to post this (unrelated to the original question)
side topic is: can those of you with experience on other platforms
tell me whether this is an inevitable consequence of virtual memory,
or just some idiotic thing that Microsoft, in their infinitessimal
wisdom didn't foresee?


Charles
-- 
http://www.enter.net/~cfusner
"...Then darkness took me, and I strayed out of thought and time,
 and I wandered far on roads that I will not tell..." 
                              -The Two Towers, JRR Tolkien


Post a reply to this message

From: Ken
Subject: Re: memory leak in 3.1g?
Date: 21 Aug 1999 18:12:32
Message: <37BF242F.85CBC8FD@pacbell.net>
Charles wrote:
> 
> Thorsten Froehlich wrote:
> > As said before, the swap file is managed by the operating system. Most
> > operating system memory managers can determine efficiently which blocks of
> > memory to move to disk and which are needed often and therefore should stay
> > in memory.
> 
> Aye, and a pity 'tis, too, considering a weird thing that just came
> to light which I'm assuming (simply on general principals) is a
> MS Windows thang...
> 
> I have a trace currently that exceeds my available memory considerably
> during parse (from what I can see parsing seems to pull two to three
> times the memory needed for the actual render itself, although it
> releases the excess when finished). So anyway, it parses for a bit,
> then starts to swap, then finally finishes parsing and returns a
> solid 30 megs of RAM, but get this... apparently the portion doled
> to swapfile is still out there on the disk for some reason. This
> conclusion I draw from the facts that A) The allocated plus free
> mem exceeds what I have installed, and B) The harddrive thrashes
> halfway to Montana and back when the trace finishes.
> 
> You would think the OS is your friend. That it would say: Hey, I've
> got 17 megs in swapfile, and 30 megs of physical RAM free; Wouldn't
> my pal, the user, be pleased if I stuck this virtual mem back where
> it belongs and boosted his performance? But alas, this doesn't
> look like the case.
> 
> What brings me to post this (unrelated to the original question)
> side topic is: can those of you with experience on other platforms
> tell me whether this is an inevitable consequence of virtual memory,
> or just some idiotic thing that Microsoft, in their infinitessimal
> wisdom didn't foresee?
> 
> Charles
> --
> http://www.enter.net/~cfusner
> "...Then darkness took me, and I strayed out of thought and time,
>  and I wandered far on roads that I will not tell..."
>                               -The Two Towers, JRR Tolkien

  This is one of my pet peaves beleive me. Windows has some funny ways of
managing memory and none of it is related to human logic. Take a look I 
believe in the advanced users group for a thread I started not too long
ago. There was a rather lenghty conversation on the pitiful waste of
system memory when having to go to the swap file and the reasons behind
it. If I can find the thread I will give you a direct pointer to it.

-- 
Ken Tyler

See my 700+ Povray and 3D Rendering and Raytracing Links at:
http://home.pacbell.net/tylereng/index.html


Post a reply to this message

From: Jon A  Cruz
Subject: Re: memory leak in 3.1g?
Date: 21 Aug 1999 19:37:13
Message: <37BF38A2.3DFFAA40@geocities.com>
Charles wrote:

> What brings me to post this (unrelated to the original question)
> side topic is: can those of you with experience on other platforms
> tell me whether this is an inevitable consequence of virtual memory,
> or just some idiotic thing that Microsoft, in their infinitessimal
> wisdom didn't foresee?

Some of it depends on which runtime libraries the application is using, which
compiler, etc.


But for Windows, consider this: They are the same people who originally hyped
NTFS as no ever needing a defragmenter.

They a few years later they hyped the fact that NT 5.0 was going to include the
NTFS defragmenting technology developed by an independent company.

--
"My new computer's got the clocks, it rocks
But it was obsolete before I opened the box" - W.A.Y.


Post a reply to this message

From: Ken
Subject: Re: memory leak in 3.1g?
Date: 21 Aug 1999 19:38:12
Message: <37BF3845.CC94A797@pacbell.net>
Ken wrote:
> There was a rather lenghty conversation on the pitiful waste of
> system memory when having to go to the swap file and the reasons behind
> it. If I can find the thread I will give you a direct pointer to it.

See the thread "Memory reporting" in povray.windows posted on 07-26-1999.

-- 
Ken Tyler

See my 700+ Povray and 3D Rendering and Raytracing Links at:
http://home.pacbell.net/tylereng/index.html


Post a reply to this message

From: Charles
Subject: Re: memory leak in 3.1g?
Date: 21 Aug 1999 22:01:21
Message: <37BF57BC.624DDADD@enter.net>
Ken wrote:
> 
> Ken wrote:
> > There was a rather lenghty conversation on the pitiful waste of
> > system memory when having to go to the swap file and the reasons behind
> > it. If I can find the thread I will give you a direct pointer to it.
> 
> See the thread "Memory reporting" in povray.windows posted on 07-26-1999.
> 

Ah, yes. Thank you very muchly. This thread did indeed go a long 
way toward answering some of my questions about this --- weird --- 
phenomenon. Looks like it it a windows thing, afterall (big 
surprise). Eventually, in the far flung reaches of what we currently 
know as the future, I'll be upgrading the whole kit and kaboodle 
(what IS a kaboodle, anyway?) and my current machine will become an 
exploration box so I have something to experiment on (hardware and 
softwarewise) without risking my prime machine. I think I just 
added "check out Linux" to my computer's list of things to do when 
it's retired<g>. In the  meantime, guess I'll just have to max out 
my memory & learn to stop gritting my teeth to the sound of hard
drive thrashing... :\


Charles
-- 
http://www.enter.net/~cfusner
"...Then darkness took me, and I strayed out of thought and time,
 and I wandered far on roads that I will not tell..." 
                              -The Two Towers, JRR Tolkien


Post a reply to this message

From: TonyB
Subject: Re: memory leak in 3.1g?
Date: 22 Aug 1999 01:13:12
Message: <37BF7813.B51B1DA6@panama.phoenix.net>
> I haven't gotten the message you describe. I do however get "File changed
> outside POV, reload?" quite often, although I've done nothing with the file in
> question.

Ditto aplenty...

--
Anthony L. Bennett
http://welcome.to/TonyB

Non nova, sed nove.


Post a reply to this message

From: Peter Popov
Subject: Re: memory leak in 3.1g?
Date: 22 Aug 1999 17:29:52
Message: <37c0db4a.99162161@204.213.191.228>
I've seen a one-month old HDD of an NT workstation. Since formatting,
NT was installed, PhotoShop, Corel Draw, Illustrator, Painter, 3DMAX,
Adobe Premierre, Cool Edit Pro (Taiti OEM), some PerceptionVR
software, and various smaller packages including POV-Ray (my doing :)
no one actually used it though :( ) This was the main workstation in a
local multimedia company. It had two HDs, one for software and one for
data. Oddly, software was working almost twice as slow as compared to
the PhotoShop-only machine or CorelDraw only machine, which were
equivalent in MB and CPU but with less memory and HD space, and were
running Windows 95 OSR 2. When the sysop ordered a trird party plug-in
for the workstation and ran it on the software HD, he almost lost his
marbles (me too.) There were about 500 file fragments, 200 swap
fragments and several thousand (!) free space fragments.

Perhaps it would have behaved better if there was more free space on
the HD. There was about a gig free, but swap was only about 50 MB at
the time (and it often ran up to 300-400 MB), I don't know. But to
claim NT needs no defragmenting is a dam^Hrn lie from MicroSoft and
nothing else.

AFAIK Linux needs no defragmenting (at least I've never seen a
defragmenter), but still when booting my SlackWare 3.5 it says "0.2%
non-contiguous" -- anyone have an idea what this might mean? I know
this part should really go to povray.unix, but then again, the first
part is for p.w :)


Peter Popov
ICQ: 15002700


Post a reply to this message

From: Mark Wagner
Subject: Re: memory leak in 3.1g?
Date: 23 Aug 1999 01:00:13
Message: <37c0d55d@news.povray.org>
Charles wrote in message <37BF57BC.624DDADD@enter.net>...
>...I'll be upgrading the whole kit and kaboodle (what IS a kaboodle,
anyway...

A kaboodle is what you get when you sell that which you acquired using your
kit.

Mark


Post a reply to this message

From: Cliff Bowman
Subject: Re: memory leak in 3.1g?
Date: 23 Aug 1999 05:22:39
Message: <37c1081e.43274206@news.povray.org>
On Fri, 20 Aug 1999 11:42:57 +0200, Remco de Korte
<rem### [at] xs4allnl> wrote:

>Dennis Miller wrote:
>> 
>> Howdy. I often do long animations that have very complex scenes, to the
>> extent that when I try to quit, it takes a few minutes to reclaim the
>> memory. However, even after quitting and closing the interface, my
>> system will grind to an incredibly slow tempo until I reboot. In fact,
>> once when I had quit PV and was trying to shut down, pvengine was still
>> showing in my Task Window (Win 98). Anyone ever have a similar problem?
>
>Yep. I think it has to do with the amount of memory that has to be recovered.
>If the memory is in the swap file you'll hear your harddisk rattling for about
>half an hour and you know what's going on and that seems only half as startling
>as when your system tries to regain control over 256Mb of Ram.
>Most of the time my system freezes for about a minute, the only thing still
>working is the mouse but every now and then just moving the mouse will cause a
>rattle from the internal speaker. Is this to simulate the harddisk?

Ot's probably your PC "reporting" (albiet somewhat uninformatively)
uninterpreted interrupts. You can get similar clicks if you've a
faulty keyboard/mouse that's generating spurious IRQ's (not much help
I know but my best guess).

>> Also, when I am rendering, hit Pause, and run another app, such as
>> Netscape, I get a series of "Cannot save file, file in use by another
>> application (paraphrased)" messages from POV. It's as if POV doesn't
>> recognize it is the app using the file.
>> Any comments?
>> Thanks much.
>> d.
>
>I see this message annoyingly often, but not really with a paused render.
>> 
You can if you pause it before POV has finished parsing. Once it's
parsed, and is building bounding blocks, or light buffers, or vista
buffers, or actually rendering then AFAIK it doesn't happen... at
least, not here!


Cheers,

Cliff Bowman
Why not pay my 3D Dr Who site a visit at http://www.who3d.cwc.net/


Post a reply to this message

<<< Previous 6 Messages Goto Initial 10 Messages

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.