 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
I had a render running that's pretty big (20GB mem use)
and it fails with a "Operation timed out" problem.
This shows up when the first frame is finished and it starts on the second frame
Render Time:
Photon Time: No photons
Radiosity Time: No radiosity
Trace Time: 0 hours 9 minutes 25 seconds (565.473 seconds)
using 8 thread(s) with 4145.640 CPU-seconds total
Rendering frame 11 of 3951
Operation timed out.
Operation timed out.
Render failed
My guess is that pov times out waiting for the swap somehow.
I found the error text in pov_err.cpp (obviously), but can't find where it's
initiated.
Running Povray 3.7 with the -CC switch on Opensuse 13.1 on a 16GB 8-core
Btw, I've had renders of similar size, and larger, running under Povray 3.6 And even
though they chewed the heck outta the swap drive, they never timed out.
--
Ger
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Ger <No.### [at] Thank You> wrote:
> I had a render running that's pretty big (20GB mem use)
> and it fails with a "Operation timed out" problem.
>
> This shows up when the first frame is finished and it starts on the second frame
>
> Render Time:
> Photon Time: No photons
> Radiosity Time: No radiosity
> Trace Time: 0 hours 9 minutes 25 seconds (565.473 seconds)
> using 8 thread(s) with 4145.640 CPU-seconds total
>
> Rendering frame 11 of 3951
>
> Operation timed out.
> Operation timed out.
> Render failed
>
> My guess is that pov times out waiting for the swap somehow.
> I found the error text in pov_err.cpp (obviously), but can't find where it's
initiated.
If swapping takes way too long, the program may end up being waiting several
seconds. There are timeouts for the internal message passing that are designed
to catch situations when the receiving thread hangs. They are 10 seconds in wall
clock time iirc. Try building with -DkDefaultTimeout=30 and see if it helps.
Thorsten
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Thorsten Froehlich wrote:
>
> If swapping takes way too long, the program may end up being waiting
> several seconds. There are timeouts for the internal message passing that
> are designed to catch situations when the receiving thread hangs. They are
> 10 seconds in wall
> clock time iirc. Try building with -DkDefaultTimeout=30 and see if it
> helps.
>
Tried with 30, raised it to 300 and it still fails. But the messages come well before
the 300 seconds are over.
Also, I kept an eye on memory usage, specially at the end of the render, and the usage
never drops until it fails.
I'll build it with some crazy value and see what happens, but it may take a while
before that one finishes. :)
--
Ger
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Am 20.01.2014 08:39, schrieb Ger:
> I had a render running that's pretty big (20GB mem use)
^^^^^^
> and it fails with a "Operation timed out" problem.
...
> My guess is that pov times out waiting for the swap somehow.
^^^^^^^^^^^^^^^^^^^^
...
> Running Povray 3.7 with the -CC switch on Opensuse 13.1 on a 16GB 8-core
^^^^^^
Bad idea (thought not nearly as bad as on a Windows machine).
> Btw, I've had renders of similar size, and larger, running under Povray 3.6
> And even though they chewed the heck outta the swap drive, they never timed out.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
As speed doesn't seem to be a matter for you, trying to lower the number
of competing threads using +WT4 or some such could be worth trying. With
the swap drive surely being the bottleneck, you might even consider
+WT1, reducing page thrashing to the absolute minimum possible in this
setup.
Note that the more worker threads you have, the less coherent
consecutive memory accesses will be.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
clipka wrote:
> Am 20.01.2014 08:39, schrieb Ger:
>> I had a render running that's pretty big (20GB mem use)
> ^^^^^^
>> and it fails with a "Operation timed out" problem.
> ...
>> My guess is that pov times out waiting for the swap somehow.
> ^^^^^^^^^^^^^^^^^^^^
> ...
>> Running Povray 3.7 with the -CC switch on Opensuse 13.1 on a 16GB 8-core
> ^^^^^^
> Bad idea (thought not nearly as bad as on a Windows machine).
>
>
>> Btw, I've had renders of similar size, and larger, running under Povray
>> 3.6 And even though they chewed the heck outta the swap drive, they never
>> timed out.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> As speed doesn't seem to be a matter for you, trying to lower the number
> of competing threads using +WT4 or some such could be worth trying. With
> the swap drive surely being the bottleneck, you might even consider
> +WT1, reducing page thrashing to the absolute minimum possible in this
> setup.
>
> Note that the more worker threads you have, the less coherent
> consecutive memory accesses will be.
Sounds logical. I'll give the +WT1 option a go and see what happens
As a side note, when the actual render starts it never drops below 95% CPU usage until
it finishes.
--
Ger
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Ger wrote:
> clipka wrote:
>
>> Am 20.01.2014 08:39, schrieb Ger:
>>> I had a render running that's pretty big (20GB mem use)
>> ^^^^^^
>>> and it fails with a "Operation timed out" problem.
>> ...
>>> My guess is that pov times out waiting for the swap somehow.
>> ^^^^^^^^^^^^^^^^^^^^
>> ...
>>> Running Povray 3.7 with the -CC switch on Opensuse 13.1 on a 16GB 8-core
>> ^^^^^^
>> Bad idea (thought not nearly as bad as on a Windows machine).
>>
>>
>>> Btw, I've had renders of similar size, and larger, running under Povray
>>> 3.6 And even though they chewed the heck outta the swap drive, they
>>> never timed out.
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> As speed doesn't seem to be a matter for you, trying to lower the number
>> of competing threads using +WT4 or some such could be worth trying. With
>> the swap drive surely being the bottleneck, you might even consider
>> +WT1, reducing page thrashing to the absolute minimum possible in this
>> setup.
>>
>> Note that the more worker threads you have, the less coherent
>> consecutive memory accesses will be.
>
> Sounds logical. I'll give the +WT1 option a go and see what happens
> As a side note, when the actual render starts it never drops below 95% CPU
> usage until it finishes.
>
No such luck
Render Time:
Photon Time: No photons
Radiosity Time: No radiosity
Trace Time: 1 hours 5 minutes 6 seconds (3906.412 seconds)
using 1 thread(s) with 3833.698 CPU-seconds total
Rendering frame 11 of 4091
Operation timed out.
Operation timed out.
Render failed
--
Ger
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Ger wrote:
> I had a render running that's pretty big (20GB mem use)
> and it fails with a "Operation timed out" problem.
>
I have cut the render back to where it's just touching swap for about 100-200MB
This is what I get now
----------------------------------------------------------------------------
Finite Objects: 12040971
Infinite Objects: 1178
Light Sources: 4
Total: 12042153
----------------------------------------------------------------------------
Parser Time
Parse Time: 0 hours 4 minutes 51 seconds (291.274 seconds)
using 1 thread(s) with 178.430 CPU-seconds total
Bounding Time: 0 hours 5 minutes 0 seconds (300.321 seconds)
using 1 thread(s) with 143.572 CPU-seconds total
----------------------------------------------------------------------------
Render Time:
Photon Time: No photons
Radiosity Time: No radiosity
Trace Time: 0 hours 10 minutes 22 seconds (622.810 seconds)
using 8 thread(s) with 4904.880 CPU-seconds total
Rendering frame 11 of 4311
----------------------------------------------------------------------------
Parser Time
Parse Time: 0 hours 12 minutes 19 seconds (739.418 seconds)
using 1 thread(s) with 290.039 CPU-seconds total
Bounding Time: 0 hours 6 minutes 45 seconds (405.536 seconds)
using 1 thread(s) with 161.738 CPU-seconds total
----------------------------------------------------------------------------
Render Options
Quality: 9
Bounding boxes.......On Bounding threshold: 3
Antialiasing.........On (Method 1, Threshold 0.010, Depth 3, Jitter 1.00,
Gamma 2.50)
Render Time:
Photon Time: No photons
Radiosity Time: No radiosity
Trace Time: 0 hours 10 minutes 41 seconds (641.523 seconds)
using 8 thread(s) with 4960.879 CPU-seconds total
Rendering frame 21 of 4311
Operation timed out.
Operation timed out.
Render failed
Only change between frames is the camera position
--
Ger
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Am 23.01.2014 22:47, schrieb Ger:
> Ger wrote:
>
>> I had a render running that's pretty big (20GB mem use)
>> and it fails with a "Operation timed out" problem.
>>
> I have cut the render back to where it's just touching swap for about 100-200MB
>
> This is what I get now
> Parser Time
> Parse Time: 0 hours 4 minutes 51 seconds (291.274 seconds)
> using 1 thread(s) with 178.430 CPU-seconds total
> Bounding Time: 0 hours 5 minutes 0 seconds (300.321 seconds)
> using 1 thread(s) with 143.572 CPU-seconds total
> ----------------------------------------------------------------------------
...
> Parser Time
> Parse Time: 0 hours 12 minutes 19 seconds (739.418 seconds)
> using 1 thread(s) with 290.039 CPU-seconds total
> Bounding Time: 0 hours 6 minutes 45 seconds (405.536 seconds)
> using 1 thread(s) with 161.738 CPU-seconds total
> ----------------------------------------------------------------------------
...
> Only change between frames is the camera position
Note how the scene already takes significantly longer to parse, which
should normally not happen if all that's changed is the camera position;
this shows how I/O performance is already degrading.
Which reminds me that there still /is/ something worth trying: One thing
that competes with swapping for I/O bandwidth is the writing of the pov
state file, which is essentially a log of rendering progress that would
be used if you abort a render and continue it later using the +C option.
The newest development version of POV-Ray 3.7 has an option to disable
creation of that file ("-CC"), which would leave more I/O bandwidth for
swapping.
Currently there is no Windows build of that version, but the newest
UberPOV beta (1.37.0.0-beta.4) also has it.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
clipka wrote:
> Note how the scene already takes significantly longer to parse, which
> should normally not happen if all that's changed is the camera position;
> this shows how I/O performance is already degrading.
>
> Which reminds me that there still /is/ something worth trying: One thing
> that competes with swapping for I/O bandwidth is the writing of the pov
> state file, which is essentially a log of rendering progress that would
> be used if you abort a render and continue it later using the +C option.
> The newest development version of POV-Ray 3.7 has an option to disable
> creation of that file ("-CC"), which would leave more I/O bandwidth for
> swapping.
>
LOL
Guess who asked for that option :)
> Currently there is no Windows build of that version, but the newest
> UberPOV beta (1.37.0.0-beta.4) also has it.
--
Ger
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Am 24.01.2014 04:45, schrieb Ger:
> clipka wrote:
>
>> Note how the scene already takes significantly longer to parse, which
>> should normally not happen if all that's changed is the camera position;
>> this shows how I/O performance is already degrading.
>>
>> Which reminds me that there still /is/ something worth trying: One thing
>> that competes with swapping for I/O bandwidth is the writing of the pov
>> state file, which is essentially a log of rendering progress that would
>> be used if you abort a render and continue it later using the +C option.
>> The newest development version of POV-Ray 3.7 has an option to disable
>> creation of that file ("-CC"), which would leave more I/O bandwidth for
>> swapping.
>>
> LOL
>
> Guess who asked for that option :)
I don't remember, um... (mumbles)...
... never mind >_<
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
