On 9/11/20 9:29 PM, jr wrote:
> "jr" <cre### [at] gmailcom> wrote:
...
> 
> and then there are a few scene files which, for one reason or other, need to be
> excluded from the run; four updates to the database required, details attached.
> 
> two scenes actually fail for v3.8, but work for 3.7.  again, see attached.
> 

We should re-start this testing discussion in a group other than off 
topic where the posts go poof after a time... (povray.beta-test ?)

When you run, are you running v3.7 shipped scenes with v3.7 povray and 
v3.8 scenes with v3.8 povray? Some scenes are different v3.7 to v3.8 to 
fix issues or align with v3.8 changes.

There are also new scenes in v3.8 where no comparison is possible. With 
these results should at least be run and looked over.

The allscene.sh script as I recall was not handling even all the v3.7 
base features and scenes, but been years since I was doing this sort of 
testing (when we pushed and dropped a v3.71).

When I run comparisons I almost always run with one thread despite this 
being slow. Otherwise some results - radiostiy involved ones for example 
- can be quite different, same version and file, run to run.

There is also the issues of all the jitter settings if you want to do 
detailed automatic comparisons - these jitters all need to be off in 
such cases. AA, area lights... Oh and you have to avoid some features 
like crand (subsurface?).

A kind of testing I don't believe has been done for v3.8 (even in the 
v3.71 testing) - except maybe by Christoph - is to render to all the 
output file formats.

Some of the outputs need a viewer (preferably not POV-Ray itself) which 
supports dithering and the higher bit depth outputs. Test here I guess 
would be do things look more or less OK output format to output format. 
We wouldn't need to run all the scenes for this testing. We might lean 
on one of the image packages to do conversions and comparisons, but 
running such programs can get pretty detailed if the outputs are not all 
linear.

Further, we should read into bump_map, image_maps all the supported 
formats (tiff for example we can read, but not write).

There was a fair bit of file io work, fixes for bugs, better support for 
some formats in v3.8.

An example of, do 'new' v3.8 things still work, testing: Do all the new 
output dithering modes still work as they should. A concern I have is 
the time (6-7 years) since some of the earlier v3.8 changes were made 
and tested. Changes since the older changes have the potential to break 
or change (for better or worse) the earlier v3.8 changes. Without 
dedicated tests for the features, which we often dont' have, we won't 
know. Suppose a lot of these fringe features for mainstream users. Maybe 
this sort of testing not at the top of the list...

Anyway. Please forgive the typos/breaks in thought. Rushing ahead of my 
first coffee.

---
IIRC with respect to those two scenes which are not running for you. The 
first is a real error and I thought the scene itself was fixed in v3.8?

With the second there was an old include file with a vertical feed or 
form feed character in it which died with Christoph's new parser 
changes. My argument was to just fix the shipped file as probably people 
should avoid such characters these days. His argument was there might be 
old files in the wild with those characters and the parser should handle 
them. What I don't know is whether he made those parser updates...

My last question is whether you are running the last v3.8 official 
pre-release or the master branch at last commit?

There were fixes and other changes in the commits between the last 
official pre-release and what is in master today. Round about way to say 
we need to decide from what "commit" we are going to try and release. I 
think many of us are running code based off the last commit (Jerome, me, 
Dick?), but maybe most are running - can only run - the last pre-release?

Bill P.

Post a reply to this message