POV-Ray : Newsgroups : povray.beta-test : File png/ppm gamma issues. v3.8. Server Time
12 May 2021 00:55:56 EDT (-0400)
  File png/ppm gamma issues. v3.8. (Message 1 to 4 of 4)  
From: William F Pokorny
Subject: File png/ppm gamma issues. v3.8.
Date: 20 Mar 2021 13:31:20
Message: <60563168$1@news.povray.org>
Recently in responding to a question about greyscale bit depths posted 
by Ingo in povray.general, I happened to notice it was impossible to set 
the grey scale gamma for output pgm/ppm files. We are in v3.8 always 
getting linear output though the bit depth changes of v3.8 appear to work.

The fix for this is to make changes the write function in ppm.cpp to 
look something like:

     // The official Netpbm standard mandates the use of the ITU-R-BT.709
     // transfer function, although it acknowledges the use of linear
     // encoding or the sRGB transfer function as alternative de-facto
     // standards.
     gamma = options.GetTranscodingGammaCurve(BT709GammaCurve::Get());

     // Greyscale or color output
     // TODO - the check for image container type is here to mimic
     // old code; do we still need it?
     if (image->IsGrayscale() || options.grayscale)
     {
         grayscale = true;
         if (plainFormat)
             file->printf("P2\n");
         else
             file->printf("P5\n");
     }
     else
     {
         if (plainFormat)
             file->printf("P3\n");
         else
             file->printf("P6\n");
     }

where the gamma setting is moved out of the else condition and above the 
if/else conditional selecting greyscale or color output. With this 
change the pgm/ppm output matches the documented v3.8 behavior and that 
for png greyscale and color outputs.

---  png gamma issue.
I've mentioned elsewhere POV-Ray makes a pretty good viewer for >8 bit 
channel output files. While using POV-Ray as viewer it generates an 
output file if not suppressed. On a whim - aka asking for trouble - I 
decided to compare the original rendered output file with the viewer 
output file - these should match. They don't.

First looking at the POV-Ray png default color output and immediate 
input as an image_map in a viewer.

The short story is png output is hard coded to srgb unless overridden by 
a File_Gamma setting somewhere. However, this hard coded output doesn't 
trigger the generation of the correct srgb, encoding, information block 
in the png file, but rather a power law 2.2 gamma block. When the 
generated image is read it's done using a gamma correction which doesn't 
match that used to generate the image.

What this means in practice is shown in the attached image top row where 
the differences are multiplied by 50. The differences are small, but 
certainly there.

In addition, looking at the output files generated by the following 
commands and using iinfo to see the _gamma block encoding I see:

---
p380 render.pov +w400 +h400 +d -p +fn8 +ofn8

     oiio:ColorSpace: "GammaCorrected2.2"
     oiio:Gamma: 2.2

---
p380 render.pov +w400 +h400 +d -p +fn8 +ofn8_srgb File_Gamma=srgb

     oiio:ColorSpace: "sRGB"

---
p380 render.pov +w400 +h400 +d -p +fn8 +ofn8_1 File_Gamma=1.0

     oiio:ColorSpace: "linear"
     oiio:Gamma: 1


---
p380 render.pov +w400 +h400 +d -p +fn8 +ofn8_bt709 File_Gamma=bt709

     oiio:ColorSpace: "GammaCorrected1.9"
     oiio:Gamma: 1

Not worked out how to update the color png gamma code as yet. Or looked 
into similar pgm/ppm differences. Suppose a good near term rule would be 
to always specify the output and input gammas you want. When I do this 
the render result and the viewer result output images match as I think 
they should.

Aside: The bottom row of the attached image shows the greyscale 
differences which include the default png gamma out issue above - plus 
apparently more...

Bill P.

---
For those wanting to play at home:

//-------- render.pov
#version 3.8; // Simple scene to test file gamma behavior.
global_settings { assumed_gamma 1 }
#declare Grey50 = srgb <0.5,0.5,0.5>;
background { color Grey50 }
#declare Camera00 = camera {
     perspective
     location <3,3,-3.001>
     sky y
     angle 35
     right x*(image_width/image_height)
     look_at <0,0,0>
}
#declare White = srgb <1,1,1>;
#declare Light00 = light_source { <50,150,-250>, White }
#declare Red = srgb <1,0,0>;
#declare CylinderX = cylinder { -1*x, 1*x, 0.01 pigment { Red } }
#declare Green = srgb <0,1,0>;
#declare CylinderY = cylinder { -1*y, 1*y, 0.01 pigment { Green } }
#declare Blue = srgb <0,0,1>;
#declare CylinderZ = cylinder { -1*z, 1*z, 0.01 pigment { Blue } }
#include "functions.inc"
#declare Fn00 = function (x,y,z) { f_sphere(x,y/3,z,0.1) }
#declare Xfrm00 = transform {
     rotate z*30.0
     translate <0.5,0.0,0.0>
}
#declare FnXfrm00 = function { transform {Xfrm00 inverse} }
#declare Fn00x = function (x,y,z) {
     Fn00(FnXfrm00(x,y,z).x,FnXfrm00(x,y,z).y,FnXfrm00(x,y,z).z)
}
#declare Xfrm01 = transform {
     rotate z*60.0
     scale <1,3,1>
}
#declare FnXfrm01 = function { transform {Xfrm01 inverse} }
#declare Fn00xx = function (x,y,z) {
     Fn00x(FnXfrm01(x,y,z).x,FnXfrm01(x,y,z).y,FnXfrm01(x,y,z).z)
}
#declare Iso99 = isosurface {
     function { Fn00xx(x,y,z) }
     contained_by { box { -2.0,2.0 } }
     threshold 0
     accuracy 0.0005
     max_gradient 1.1
     pigment { color Green }
}

//--- scene ---
     camera { Camera00 }
     light_source { Light00 }
     object { CylinderX }
     object { CylinderY }
     object { CylinderZ }
     object { Iso99 }


//-------- viewer.pov
// p380 viewer.pov +w400 +h400 +d +p +fn8
// Replace three fn8.png strings below with whatever.
#version 3.8;
global_settings { assumed_gamma 1 }
#declare Grey50 = srgb <0.5,0.5,0.5>;
background { color Grey50 }
#declare VarOrthoMult =
     1.0/max(image_width/image_height,image_height/image_width);
#declare Camera01z = camera {
     orthographic
     location <0,0,-2>
     direction z
     right VarOrthoMult*x*max(1,image_width/image_height)
     up VarOrthoMult*y*max(1,image_height/image_width)
}
#macro ImageMap00(_HF)
   #if (_HF=0)
     image_map { "fn8.png"
   #else
      "fn8.png"
   #end
   #if (_HF=0)
      map_type 0
      once
    //interpolate 2 // No interpolation
     }
   #end
#end
#declare ImageMap00_P = pigment { image_map { "fn8.png" gamma 1 } }
#declare ImageMap00_Range = max_extent(ImageMap00_P);
#declare ImageMap00_NrmScale =
     <min(1,ImageMap00_Range.x/ImageMap00_Range.y),
      min(1,ImageMap00_Range.y/ImageMap00_Range.x),1>;
#declare Pigment00 = pigment {
     ImageMap00(0)
     translate <-0.5,-0.5,0>
     scale ImageMap00_NrmScale
}
#declare Fin00 = finish {
     ambient 0.0
     diffuse 0
     emission srgbft <1,1,1,0,0>
}
#declare Txtr00 = texture { pigment { Pigment00 } finish { Fin00 } }
#declare Plane00 = plane { z, 0 }
#declare Obj00 = object { Plane00 texture { Txtr00 } }

//--- scene ---
     camera { Camera01z }
     object { Obj00 }


Post a reply to this message


Attachments:
Download 'png_gamma_story.png' (33 KB)

Preview of image 'png_gamma_story.png'
png_gamma_story.png


 

From: William F Pokorny
Subject: Re: File png/ppm gamma issues. v3.8.
Date: 22 Mar 2021 07:34:52
Message: <605880dc$1@news.povray.org>
On 3/20/21 1:31 PM, William F Pokorny wrote:
...
> 
> ---  png gamma issue.
...
> 
With respect to the current v3.8 not writing a png srgb gamma block 
unless somewhere File_Gamma is set to 'srgb', the current v3.7 looks to 
be OK. If the gamma correction used was srgb, explictly or not, the srgb 
block is always written.

Somewhere during the v3.71 / v3.8 work something changed - but I cannot 
pick up the cause just reading the code. And! this time I'm not going to 
spend 3-4 hours compiling different historical commits to try which 
commit broke things.

The fix in the png.cpp file is to change:

#if defined(PNG_WRITE_sRGB_SUPPORTED)
     if (options.encodingGamma &&
             typeid(*options.encodingGamma) == typeid(SRGBGammaCurve))
         png_set_sRGB(png_ptr, info_ptr, PNG_sRGB_INTENT_PERCEPTUAL);
#endif // PNG_WRITE_gAMA_SUPPORTED

to read:

#if defined(PNG_WRITE_sRGB_SUPPORTED)
     if ((!options.encodingGamma) ||
         (options.encodingGamma &&
          typeid(*options.encodingGamma) == typeid(SRGBGammaCurve)))
     {
         png_set_sRGB(png_ptr, info_ptr, PNG_sRGB_INTENT_PERCEPTUAL);
     }
#endif // PNG_WRITE_gAMA_SUPPORTED


With this update above the color render and viewer v3.8 outputs match 
whether File_Gamma explicitly set to srgb or not.

The grey scale too is better as shown in the attached image. The top row 
less the fix above and the bottom row with the fix. There is still 
something off related to non-grey defined colors in the render.pov 
result leading to different between render and viewer by as much as 
11/255(1). I don't understand what is happening there as yet...

(1) - Prior to the srgb gamma chunk fix the max color differences where 
9/255 at very low luminance.

With the previously posted grey output pgm/ppm fix, both the color and 
greyscale ppm outputs match so long as 'gamma=bt709' is used when 
defining the image_map (reading the POV-Ray created image file). The 
netpbm format doesn't encode the gamma on write - so you have no choice 
but to be explicit on the read if you want to be sure you match.

Aside 1: Don't trust ImgeMagick's identify command with respect to the 
srgb block being there or not. It's wrong when I run it. The iinfo 
command from the OpenImageIO package works OK. Or you can just edit the 
generated png with a text editor; this looks really ugly but if the sRGB 
chunk is being written you'll see 'sRGB' between the readable text 
'gAMA' and 'sBIT' right at the top. (Gimp, for me, is also not reporting 
all the meta data completely/correctly)

Aside 2: For those running stock v37/v38, I guess my recommendation 
would be to use linear gamma explicitly coded where you want to write an 
image file you'll later use (read) within POV-Ray. If only writing a png 
file, and using stock v3.8 be sure to use File_Gamma=srgb, if you want 
to be sure you have the srgb gamma block for downstream tools. Remember 
too Christoph corrected quite a lot post v3.7 in how the file gamma's 
are handled, but for color at least it looks to me like v3.7 png output 
OK. V3.7 image file input is another matter.

More on the png greyscale (single channel png) differences when I 
decipher more.

Bill P.


Post a reply to this message


Attachments:
Download 'pnggreymoretodo.png' (21 KB)

Preview of image 'pnggreymoretodo.png'
pnggreymoretodo.png


 

From: William F Pokorny
Subject: Re: File png/ppm gamma issues. v3.8.
Date: 25 Mar 2021 02:48:29
Message: <605c323d@news.povray.org>
On 3/22/21 7:34 AM, William F Pokorny wrote:
...
> 
> More on the png greyscale (single channel png) differences when I 
> decipher more.
> 

OK. In png.cpp there is a conditional block starting with:

...
else if ((color_type == PNG_COLOR_TYPE_GRAY) && (bit_depth <= 8))
...

and near the bottom of the block there is the conditional:

ImageDataType imagetype = options.itype;
if (imagetype == ImageDataType::Undefined)
{
     imagetype = has_alpha ?
	ImageDataType::GrayA_Int8 : ImageDataType::Gray_Int8;
}

and this should read:

ImageDataType imagetype = options.itype;
if (imagetype == ImageDataType::Undefined)
{
     imagetype = ImageDataType::Colour_Map;
}

The code is creating a color map type image in the way the conditional 
block previous does for 'actual' color mapped png images.

The bug is sitting there in v3.7 too, but I guess because greyscale 
output in v3.7 is 16 bit we don't trip it.

What this means is a valid work around for v3.8 based users is to 
specify bit depths >8 (9..16) greyscale out as that code appears to work OK.

Detail:
-------
That conditional block in question looks to be an attempt to 'pretend' 
<= 8 bit gray images are palleted images. It allows each of the 256 
valid greys to be gamma converted once instead of by pixel. The trouble 
is in setting the image type to either of those Gray*Int8 types, we end 
up getting the 'image' class's Image::SetIndexedValue method instead of 
the ColourMapImage class's Image::SetIndexedValue method (got to love c++).

This results in the incoming grey values not being used as index values 
into the created faked color map directly, but rather as indexes into 
code setting explicit RGBA values using the color map. The mystery to me 
is partly how the class mechanisms work at all(1) here and further that 
the result is as close as it is.

(1) - The compile should probably die here, but as the classes set up, 
somehow things don't.

There are some further questions too with this conditional block... 
First, for images with no mapped alpha channels, or single transparent 
color, the block isn't needed. It can be commented and the code already 
handling the >8 bit depths works too for <=8 bit depths with 'perhaps' 
some performance impact.

There are png encodings which can mark a particular color as transparent 
and this block 'perhaps' can handle some of them - though who knows. It 
looks to me like there is no actual png palette grey encoding, but 
rather only a color one. Further, the >8 bit grey scale READ code in 
POV-Ray does not handle single transparent colors that I can see. So 
guess on the surface, looks to me like grey encoding of more specialized 
palette or value transparency is iffy.

In my own code I've also added:

// PNG variously supports bit depths of 1,2,4,8 and 16. Below only
// depths of 8 and 16 look reliable to me.
POV_IMAGE_ASSERT((bit_depth == 8) || (bit_depth == 16));

to see if I turn up some of the odd encodings at some point and then I 
can perhaps look at what's happening in those cases.

(The above is the storage channel depth, not the 'value' bit depth 
(bucketing) v3.8 now allows to be set 1..16.)

There are in the current png library, methods which promote odd 
encodings and grey encodings to 8 or 16 bit rgb(a). Perhaps it's 
possible to make the current png read code simpler and more robust - if 
'perhaps' not quite as fast in some cases.

Lastly
------
Yes, this last bug a bit off in the weeds as most people writing grey 
pngs for height fields will probably use 16 bits. It's only those 
wanting to read <=8 bit png grey scale images.

Bill P.


Post a reply to this message

From: ingo
Subject: Re: File png/ppm gamma issues. v3.8.
Date: 26 Mar 2021 03:35:05
Message: <XnsACF957531BEE0seed7@news.povray.org>
in news:605c323d@news.povray.org William F Pokorny wrote:

> OK.

Thanks for investigating Bill,

Ingo


Post a reply to this message

Copyright 2003-2021 Persistence of Vision Raytracer Pty. Ltd.