POV-Ray : Newsgroups : povray.beta-test : v3.7 v3.8 image compare Server Time: 27 Sep 2020 20:13:37 GMT
  v3.7 v3.8 image compare (Message 1 to 4 of 4)  
From: jr
Subject: v3.7 v3.8 image compare
Date: 12 Sep 2020 12:15:00
Message: <web.5f5cbb11fe8c68704d00143e0@news.povray.org>
hi,

I've simply cut'n'paste your reply here.

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 9/11/20 9:29 PM, jr wrote:
> > "jr" <cre### [at] gmailcom> wrote:
> ...
> > two scenes actually fail for v3.8, but work for 3.7.  ...
> ...
> When you run, are you running v3.7 shipped scenes with v3.7 povray and
> v3.8 scenes with v3.8 povray? Some scenes are different v3.7 to v3.8 to
> fix issues or align with v3.8 changes.

no.  I understood CC to mean that one set of files gets compared with two
different executables.  only v3.7 '/scenes/' scenes are used.

> There are also new scenes in v3.8 where no comparison is possible. With
> these results should at least be run and looked over.

that would only mean replacing a few paths and the names of the executables,
more below.

at this point, as mentioned, it's v basic, no error handling, nothing, for now
I've simply .. thrown in a possible framework to see whether there's an use for
it.

> The allscene.sh script as I recall was not handling even all the v3.7
> base features and scenes, but been years since I was doing this sort of
> testing (when we pushed and dropped a v3.71).

the '/scripts/' stuff does not use the 'recommended' settings from the scene,
afaik.

> When I run comparisons I almost always run with one thread despite this
> being slow. Otherwise some results - radiostiy involved ones for example
> - can be quite different, same version and file, run to run.
>
> There is also the issues of all the jitter settings if you want to do
> detailed automatic comparisons - these jitters all need to be off in
> such cases. AA, area lights... Oh and you have to avoid some features
> like crand (subsurface?).

thank you!  will incorporate '-j' and think about '-wt'; optional perhaps?

there is no analysis of the (given) command-line options.  I'm thinking (and
would prefer) a simple UI utility to manipulate individual scene settings in the
database.
(directories can be enabled/disabled, which may be handy)

> A kind of testing I don't believe has been done for v3.8 (even in the
> v3.71 testing) - except maybe by Christoph - is to render to all the
> output file formats.

ok.

> Some of the outputs need a viewer (preferably not POV-Ray itself) which
> supports dithering and the higher bit depth outputs. Test here I guess
> would be do things look more or less OK output format to output format.
> We wouldn't need to run all the scenes for this testing. We might lean
> on one of the image packages to do conversions and comparisons, but
> running such programs can get pretty detailed if the outputs are not all
> linear.

that's for actual image compare.  there may also be milage in extracting
information from the output of, for instance, (ImageMagick) 'identify -verbose'.
 :-)

[v3.8 stuff snipped]

> Anyway. Please forgive the typos/breaks in thought. Rushing ahead of my
> first coffee.

looked productive to me.  :-)

> ...
> My last question is whether you are running the last v3.8 official
> pre-release or the master branch at last commit?

no, it's an 'unofficial' alpha.10064268.  re the below, would consider using a
more up-to-date version.

> There were fixes and other changes in the commits between the last
> official pre-release and what is in master today. Round about way to say
> we need to decide from what "commit" we are going to try and release. I
> think many of us are running code based off the last commit (Jerome, me,
> Dick?), but maybe most are running - can only run - the last pre-release?

again, although the scripts are (as yet) in no fit state, using a different
version of POV-Ray will only need changing a file name, and to use scenes
different from v3.7 also is just a matter of adjusting paths.  ideally those
data (names of exectuables, etc) ought to be in the db too; but then we'd need
to talk (re)design "properly".


regards, jr.


Post a reply to this message

From: jr
Subject: Re: v3.7 v3.8 image compare
Date: 17 Sep 2020 12:10:00
Message: <web.5f6350dc9b9dedbc4d00143e0@news.povray.org>
"jr" <cre### [at] gmailcom> wrote:
> William F Pokorny <ano### [at] anonymousorg> wrote:
> > ...
> > When I run comparisons I almost always run with one thread despite this
> > being slow. Otherwise some results - radiostiy involved ones for example
> > - can be quite different, same version and file, run to run.
> >
> > There is also the issues of all the jitter settings if you want to do
> > detailed automatic comparisons - these jitters all need to be off in
> > such cases. AA, area lights... Oh and you have to avoid some features
> > like crand (subsurface?).
>
> thank you!  will incorporate '-j' and think about '-wt'; optional perhaps?

I've made several changes and the whole thing is now a two-step process: create
a db, and run it.

see 'wtlimit'.  :-)

the archive contains a 'readme.txt' detailing the few edits needed to "install"
the code.  and I meant to mention in the 'readme' - the work directory must not
exist, ie must be deleted between successive runs.

<https://drive.google.com/file/d/10YPGxnYy0v5ph5DmNUJuYuq1NS_MOvUn/view?usp=sharing>

regards, jr.


Post a reply to this message

From: jr
Subject: Re: v3.7 v3.8 image compare
Date: 17 Sep 2020 17:20:00
Message: <web.5f6399d79b9dedbc4d00143e0@news.povray.org>
"jr" <cre### [at] gmailcom> wrote:
> ...
> the archive contains a 'readme.txt' detailing ...

I also forgot to include a whole paragraph regarding results, plus found typos.
I copy the amended 'readme' below, rather than posting another archive.

btw, to make the s/ware work with 'povr', requires the '/scenes/' directory to
contain directories only.  move 'biscuit.pov' into a sub-directory.  create a
'previews' directory (that is the "marker" the script looks for).  (I checked it
works, albeit with two identical 'povr' programs)


regards, jr.

# ------------------------------------------------------------------------
setting up, by script.

you will neeed to create two directories, one for the scripts, one for
the database(s); you could make both the same.

extract the archive in the script directory.

- cimkdb.sh

  the directories in lines 10 and 11 need to be changed to the above.
  'store_db' for the databases, 'store_ec' for external code/scripts.

  in line 34 you can change the "preferred" work directory for when
  jobs get run.  I use '/tmp/cmpimg/' because '/tmp/' is set up to
  "live" in RAM.

- cirundb.sh

  change 'store_ec' in line 11.

  the script always creates an archive of the run's data.  if that is
  unwanted, you could insert an 'exit 0' before line 66.

- init-db.sql

  there are a couple of defaults you may wish to adjust.  currently,
  jobs run without showing a display/render window, without limit on
  the number of worker threads, and generating log information; also,
  the set of "standard" (POV-Ray) CLI options is given here.
  see the comments for table 'misckv' to make necessary changes.

- mkcmpimg.tcl

  you will need to provide the names of the two POV-Ray executables
  which are to be compared, and you may need/want to use different
  "tags" to distinguish them.

  lines 14 and 15 contain the relevant data, both two-element lists.
  the tags are used as prefixes for the (temporary) image files.


how it works.

it is now a two step process.  create a database for a given POV-Ray
distribution '/scenes/' directory, and "run the database". in between,
you may wish to update individual scenes in the db, eg if specific
option switches are required.  the db files are always named with the
version number grabbed from the directory, but they can be renamed if
necessary.

for example (from the script directory):

  $ ./cimkdb.sh /usr/local/share/povray-3.7/scenes/

(an override for the default work directory can be given as second arg)

the database can now be run as often as required.

  $ ./cirundb.sh ~/.local/share/cmpimg/civ3.7.sqlite

you will have to substitute the path of course.

the results are left in the work directory.
if logging was not disabled, there will be two new files for every
"active" scene, eg 'biscuit.pov' now has 'ci_biscuit.png', output by
the 'montage' command, and 'ci_biscuit.txt.xz', which is POV-Ray's
'all_file' output, compressed.  (view with 'xzless')
a compressed archive of the work directory too is left in the work
directory.  it is named 'ciYYMMDDHHMM.tar.xz', where the date/time is
from when 'cirundb.sh' was started.

the work directory must be deleted before the next run.

run either script without any arguments to see the required command
line, including the options.

finally, there is, as yet, no documentation (although all scripts are
commented), and there is very little error handling, and no attempt at
making the scripts "resiliant".

a simple (GUI) frontend for database maintenance is (still) on the
drawing board; it is not high priority since everything can be done
easily using the 'sqlite3' shell.


Post a reply to this message

From: jr
Subject: Re: v3.7 v3.8 image compare
Date: 23 Sep 2020 16:45:00
Message: <web.5f6b7b339b9dedbc4d00143e0@news.povray.org>
"jr" <cre### [at] gmailcom> wrote:
> ...
> I've made several changes and the whole thing is now a two-step process: create
> a db, and run it.

more changes.  "installing" is a matter of setting a couple of directories and
listing available POV-Ray programs, essentially.  once set up, the POV-Rays and
database to use, plus a few runtime options, are chosen from an UI which
executes the "job".  comments and feedback, please.

the new archive is at:
<https://drive.google.com/file/d/10_S7EN-_3zm7wezQTBLVYeC79m2PUM3V/view?usp=sharing>


regards, jr.


Post a reply to this message

Copyright 2003-2008 Persistence of Vision Raytracer Pty. Ltd.