On 9/11/20 9:29 PM, jr wrote:
> "jr" <cre### [at] gmailcom> wrote:
> and then there are a few scene files which, for one reason or other, need to be
> excluded from the run; four updates to the database required, details attached.
> two scenes actually fail for v3.8, but work for 3.7. again, see attached.
We should re-start this testing discussion in a group other than off
topic where the posts go poof after a time... (povray.beta-test ?)
When you run, are you running v3.7 shipped scenes with v3.7 povray and
v3.8 scenes with v3.8 povray? Some scenes are different v3.7 to v3.8 to
fix issues or align with v3.8 changes.
There are also new scenes in v3.8 where no comparison is possible. With
these results should at least be run and looked over.
The allscene.sh script as I recall was not handling even all the v3.7
base features and scenes, but been years since I was doing this sort of
testing (when we pushed and dropped a v3.71).
When I run comparisons I almost always run with one thread despite this
being slow. Otherwise some results - radiostiy involved ones for example
- can be quite different, same version and file, run to run.
There is also the issues of all the jitter settings if you want to do
detailed automatic comparisons - these jitters all need to be off in
such cases. AA, area lights... Oh and you have to avoid some features
like crand (subsurface?).
A kind of testing I don't believe has been done for v3.8 (even in the
v3.71 testing) - except maybe by Christoph - is to render to all the
output file formats.
Some of the outputs need a viewer (preferably not POV-Ray itself) which
supports dithering and the higher bit depth outputs. Test here I guess
would be do things look more or less OK output format to output format.
We wouldn't need to run all the scenes for this testing. We might lean
on one of the image packages to do conversions and comparisons, but
running such programs can get pretty detailed if the outputs are not all
Further, we should read into bump_map, image_maps all the supported
formats (tiff for example we can read, but not write).
There was a fair bit of file io work, fixes for bugs, better support for
some formats in v3.8.
An example of, do 'new' v3.8 things still work, testing: Do all the new
output dithering modes still work as they should. A concern I have is
the time (6-7 years) since some of the earlier v3.8 changes were made
and tested. Changes since the older changes have the potential to break
or change (for better or worse) the earlier v3.8 changes. Without
dedicated tests for the features, which we often dont' have, we won't
know. Suppose a lot of these fringe features for mainstream users. Maybe
this sort of testing not at the top of the list...
Anyway. Please forgive the typos/breaks in thought. Rushing ahead of my
IIRC with respect to those two scenes which are not running for you. The
first is a real error and I thought the scene itself was fixed in v3.8?
With the second there was an old include file with a vertical feed or
form feed character in it which died with Christoph's new parser
changes. My argument was to just fix the shipped file as probably people
should avoid such characters these days. His argument was there might be
old files in the wild with those characters and the parser should handle
them. What I don't know is whether he made those parser updates...
My last question is whether you are running the last v3.8 official
pre-release or the master branch at last commit?
There were fixes and other changes in the commits between the last
official pre-release and what is in master today. Round about way to say
we need to decide from what "commit" we are going to try and release. I
think many of us are running code based off the last commit (Jerome, me,
Dick?), but maybe most are running - can only run - the last pre-release?
Post a reply to this message