I create a series of simple 4K scenes with 16/32 cores/threads. Each takes 10s
from which the parsing process is <1s, and the rendering process is 9s.

In the rendering time, my CPU threads are used in the first 3s (technically, 1s
as it is a peak). This is exactly the period in which the temporary file
"pov-state" exists.

It takes about 5s to save output.png when almost no CPU resource is used.
Obviously, I hope to reduce this period.

Since the file is small 5-7MB, disk speed cannot be the bottleneck, but I
checked the saving with an SSD and no difference.

I use v3.8 on Ubuntu, and my command is

povray +Iinput.pov -GA +Q11 +MB +UL +UV +A -D -V +TH +W3840 +H2160 +UA
+Ooutout.png

Alternatively, do you have any idea for parallelising multiple processes? In the
above scenario, I have enough resources to run 3-5 POV-Ray commands at the same
time, but the peaks may overlap.

Memory is not an issue at all, as I have abundant, and as you can tell the scene
is not memory-intensive.

Post a reply to this message

"Kima" <nomail@nomail> wrote:
>
> Since the file is small 5-7MB, disk speed cannot be the bottleneck, but I
> checked the saving with an SSD and no difference.

If this is small, what do you consider large?

Post a reply to this message

Le 2020-07-10 à 00:19, Kima a écrit :
> I create a series of simple 4K scenes with 16/32 cores/threads. Each takes 10s
> from which the parsing process is <1s, and the rendering process is 9s.
> 
> In the rendering time, my CPU threads are used in the first 3s (technically, 1s
> as it is a peak). This is exactly the period in which the temporary file
> "pov-state" exists.
> 
> It takes about 5s to save output.png when almost no CPU resource is used.
> Obviously, I hope to reduce this period.
> 

Those 5 seconds are almost entirely consecrated to the encoding and 
compression of the PNG file with only a small bit actually used to 
access the drive. It's almost all single thread, sequential data access, 
analysis and manipulation.
I'm afraid that it can't get reduced by much, unless you are willing to 
reduce the compression factor, resulting in a larger file.

Post a reply to this message

"Cousin Ricky" <rickysttATyahooDOTcom> wrote:
> "Kima" <nomail@nomail> wrote:
> >
> > Since the file is small 5-7MB, disk speed cannot be the bottleneck, but I
> > checked the saving with an SSD and no difference.
>
> If this is small, what do you consider large?

With any modern HDD, writing 5-7MB of data occurs within a fraction of a second.

Post a reply to this message

Alain Martel <kua### [at] videotronca> wrote:
> Le 2020-07-10 à 00:19, Kima a écrit :
> > I create a series of simple 4K scenes with 16/32 cores/threads. Each takes 10s
> > from which the parsing process is <1s, and the rendering process is 9s.
> >
> > In the rendering time, my CPU threads are used in the first 3s (technically, 1s
> > as it is a peak). This is exactly the period in which the temporary file
> > "pov-state" exists.
> >
> > It takes about 5s to save output.png when almost no CPU resource is used.
> > Obviously, I hope to reduce this period.
> >
>
> Those 5 seconds are almost entirely consecrated to the encoding and
> compression of the PNG file with only a small bit actually used to
> access the drive. It's almost all single thread, sequential data access,
> analysis and manipulation.
> I'm afraid that it can't get reduced by much, unless you are willing to
> reduce the compression factor, resulting in a larger file.

You are right! I was able to reduce this period by changing the output format.
It is not a matter of compression but data handling since the shortest time was
to save as a lossy JPEG.

Anyhow, I should find a way to do this single-thread process in the background
while starting next rendering.

Post a reply to this message

Alain Martel <kua### [at] videotronca> wrote:
> Le 2020-07-10 à 00:19, Kima a écrit :
> > I create a series of simple 4K scenes with 16/32 cores/threads. Each takes 10s
> > from which the parsing process is <1s, and the rendering process is 9s.
> >
> > In the rendering time, my CPU threads are used in the first 3s (technically, 1s
> > as it is a peak). This is exactly the period in which the temporary file
> > "pov-state" exists.
> >
> > It takes about 5s to save output.png when almost no CPU resource is used.
> > Obviously, I hope to reduce this period.
> >
>
> Those 5 seconds are almost entirely consecrated to the encoding and
> compression of the PNG file with only a small bit actually used to
> access the drive. It's almost all single thread, sequential data access,
> analysis and manipulation.
> I'm afraid that it can't get reduced by much, unless you are willing to
> reduce the compression factor, resulting in a larger file.

As an update, if it might be beneficial to others.

In my experience, the slowest format to save is PNG. Of course, PNG is the best
and common format in terms of quality and size.

Saving in JPEG is the fastest though it is highly compressed. Unfortunately, a
heavily lossy format like JPEG is not suitable for the purpose of ray tracing.

JPEG with 100% quality is close to PNG (POV-Ray see 85% by default). The
official document emphasises that even with 100% quality, there might be some
artefacts. I am inspecting this failure but haven't found any yet. JPEG with
100% quality is about 40% of PNG size.

As a side note, I believe the default value of compression in POV-Ray (85%) is
too low. The default value in ImageMagick for conversion to JPEG is 92%. It does
not make sense to use many resources to ray trace a scene and save it as a
heavily lossy format.

Post a reply to this message

On 7/11/20 12:17 AM, Kima wrote:
...
> As an update, if it might be beneficial to others.
> 
> In my experience, the slowest format to save is PNG. Of course, PNG is the best
> and common format in terms of quality and size.
> 
> Saving in JPEG is the fastest though it is highly compressed. Unfortunately, a
> heavily lossy format like JPEG is not suitable for the purpose of ray tracing.
> 
> JPEG with 100% quality is close to PNG (POV-Ray see 85% by default). The
> official document emphasises that even with 100% quality, there might be some
> artefacts. I am inspecting this failure but haven't found any yet. JPEG with
> 100% quality is about 40% of PNG size.
> 
> As a side note, I believe the default value of compression in POV-Ray (85%) is
> too low. The default value in ImageMagick for conversion to JPEG is 92%. It does
> not make sense to use many resources to ray trace a scene and save it as a
> heavily lossy format.
> 

Thanks for the information. I found myself thinking about your post... 
Where did exr output come out size and performance wise?

There are formats other than exr which don't compress on output - ppm is 
another I think, and you can set the depth there to 16 bits per color 
channel too. If the uncompressed output is much faster, perhaps use one 
of those formats and compress/convert with external scripts?

I'm no image format expert (Christoph was the guy there) - and I haven't 
spent much time in the image file handling code. Still, I'll risk adding 
a few POV-Ray code related comments. :-)

- The jpeg output doesn't dither which may be part of the speed 
difference relative to png.

- The jpeg library itself looks to support a no-compression jpeg output 
by setting the quality to 0, but the current POV-Ray wrap maps <=1 
values (the default compression value is -1) to 85%.

- I'd say 85% jpeg compression is sometimes suitable. :-) Just depends 
on what one is doing with POV-Ray. Aside: IIRC 85% was chosen to line up 
with the effective compression relative to some previous jpeg output, 
not because it was considered the best quality choice.

- POV-Ray mostly uses standard external libraries when reading and 
writing images so the compression de-compression isn't POV-Ray per sa.
The compression approaches are not all the same - with higher quality 
and lossless compression tending to be the most expensive. Which you 
probably knew...

Aside:

The jpeg library efforts are somewhat fractured. Most support no 
compression option I believe, some lossless too, etc. There is though 
one library better optimized for modern CPUs (turbo something?) and it's 
not currently the library used by POV-Ray. I think google did much of 
the work IIRC. A bit of a tangle in any case.

Somewhere (github?) I ran across a highly optimized for simd 
architectures png library with benchmarks that looked great compared to 
the common png library. But, during my glance, it wasn't clear how much 
real world use of it there was.

Anyway... I guess, it's somewhat meaningless practically, but there are 
image libraries out there which - perhaps - could enable better image 
input/output performance in POV-Ray.

Bill P.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 7/11/20 12:17 AM, Kima wrote:
> ...
> > As an update, if it might be beneficial to others.
> >
> > In my experience, the slowest format to save is PNG. Of course, PNG is the best
> > and common format in terms of quality and size.
> >
> > Saving in JPEG is the fastest though it is highly compressed. Unfortunately, a
> > heavily lossy format like JPEG is not suitable for the purpose of ray tracing.
> >
> > JPEG with 100% quality is close to PNG (POV-Ray see 85% by default). The
> > official document emphasises that even with 100% quality, there might be some
> > artefacts. I am inspecting this failure but haven't found any yet. JPEG with
> > 100% quality is about 40% of PNG size.
> >
> > As a side note, I believe the default value of compression in POV-Ray (85%) is
> > too low. The default value in ImageMagick for conversion to JPEG is 92%. It does
> > not make sense to use many resources to ray trace a scene and save it as a
> > heavily lossy format.
> >
>
> Thanks for the information. I found myself thinking about your post...
> Where did exr output come out size and performance wise?
>
> There are formats other than exr which don't compress on output - ppm is
> another I think, and you can set the depth there to 16 bits per color
> channel too. If the uncompressed output is much faster, perhaps use one
> of those formats and compress/convert with external scripts?
>
> I'm no image format expert (Christoph was the guy there) - and I haven't
> spent much time in the image file handling code. Still, I'll risk adding
> a few POV-Ray code related comments. :-)
>
> - The jpeg output doesn't dither which may be part of the speed
> difference relative to png.
>
> - The jpeg library itself looks to support a no-compression jpeg output
> by setting the quality to 0, but the current POV-Ray wrap maps <=1
> values (the default compression value is -1) to 85%.
>
> - I'd say 85% jpeg compression is sometimes suitable. :-) Just depends
> on what one is doing with POV-Ray. Aside: IIRC 85% was chosen to line up
> with the effective compression relative to some previous jpeg output,
> not because it was considered the best quality choice.
>
> - POV-Ray mostly uses standard external libraries when reading and
> writing images so the compression de-compression isn't POV-Ray per sa.
> The compression approaches are not all the same - with higher quality
> and lossless compression tending to be the most expensive. Which you
> probably knew...
>
> Aside:
>
> The jpeg library efforts are somewhat fractured. Most support no
> compression option I believe, some lossless too, etc. There is though
> one library better optimized for modern CPUs (turbo something?) and it's
> not currently the library used by POV-Ray. I think google did much of
> the work IIRC. A bit of a tangle in any case.
>
> Somewhere (github?) I ran across a highly optimized for simd
> architectures png library with benchmarks that looked great compared to
> the common png library. But, during my glance, it wasn't clear how much
> real world use of it there was.
>
> Anyway... I guess, it's somewhat meaningless practically, but there are
> image libraries out there which - perhaps - could enable better image
> input/output performance in POV-Ray.
>
> Bill P.

Thanks for the detailed description.

As the number of available cores/threads, it is vital to reduce this
single-thread process when creating many scenes (e.g., for animation). I am
experimenting with different cases to better understand possible outlets.

Compression is not the key factor. In the case of tga, it is only 40% faster to
use the uncompressed output as compared with the compressed one.

ppm and exr both are faster than png but not close to JPEG.

I reckon a key factor (if not the main) is the absence of alpha channel in JPEG.

Quality 0 will lead to the default 85%. Non-compression is the quality of 100%.
Technically, it is not compressed by the JPEG standard, but it is still
compressed because of the JPEG architecture algorithm (leading to the
possibility of artefacts).

I am trying to find out the chance of artefacts when using JPEG. If it is
negligible, then I believe JPEG 100% is a better choice than PNG when
transparency is not required.

Post a reply to this message

I really haven't looked into graphics file formats in a long time, so I'll just
ask this and see what happens.  ;)

What about BMP or GIF?
I also never see anyone use RAW file formats - except for professional
photographers, and it seems a bit --- complex.

Post a reply to this message

"Bald Eagle" <cre### [at] netscapenet> wrote:
> I really haven't looked into graphics file formats in a long time, so I'll just
> ask this and see what happens.  ;)
>
> What about BMP or GIF?
> I also never see anyone use RAW file formats - except for professional
> photographers, and it seems a bit --- complex.

RAW is technically useless (at least not beneficial). In a digital camera, RAW
formats store the exact information received by the sensor. Then, the image is
simulated from the lights sensed by the sensor.

POV-Ray simulates the color of each pixel by ray tracing. For this reason,
POV-Ray has dropped supporting RAW.

BMP like PPM and TGA is a binary map of pixels (bitmap). PNG has the same
information (almost) but with better size management and zip-compression (not
affecting the information stored).

The best format (in terms of size and accuracy) is PNG.

Of course, POV-Ray is not slow in producing PNG. As I tested, converting an
uncompressed format to PNG takes the same time. As Bill correctly mentioned,
POV-Ray uses other libraries for producing each image format.

Post a reply to this message