I'd been posting about some of the povr branch stuff I thought perhaps 
useful to an eventual v3.8 or V4.0 elsewhere, but given it looks like 
v4.0 will be starting up as an actual development branch I'm going to 
start to posting the potential 'povr ideas' for v4.0 here.

--- On better parser checking
An idea I've recently started to code up in the povr branch is better 
parser SDL checking around function and vectors. It's been the case that 
not nearly everything which could be checked by the parser is so a lot 
can slip by. As the example which comes to mind the pow() function only 
parser domain check is whether both input values are zero.

Checking everything is a not insignificant expense at parse time. So, 
the approach I've taken is the introduction of a compile time 
configuration called POV_PARSER_SDL_DEBUG. By default the few parse time 
checks are now off. When POV_PARSER_SDL_DEBUG the old checks and many 
more are enabled.

POV_PARSER_SDL_DEBUG is also activated when the more general POV_DEBUG 
is set which itself activates a bunch of internal code assertions. When 
I looked at the standard POV-Ray debian packaging on my move to Ubuntu 
20.04, I noticed in the debian developer space private work around 
providing debug versions of POV-Ray in addition to the standard version 
(probably with debug symbols too). Providing a normal version of 
POV-Ray, and a debug version, I think it a good idea.

So, it would be that on seeing something funky happening, a standard 
practice would be to run the debug version of POV-Ray over the normal. 
If lucky, the 'POV_DEBUG' version will point directly to the issue 
where, POV-Ray today, mostly lets numerical stuff go - slightly more 
true on unix/linux where compiling with -ffast-math.

--- The new parser checking and the color vector being at float.
While checking some of the new POV_PARSER_SDL_DEBUG I stumbled my way to 
the following test / test results. First, what one gets from the 
particular tests today in v3.7 or v3.8.

    ---
V2.x*1 -->  3e38 300000000000000012135895401846682943488.000000
V2.y*1 -->  5e38 499999999999999969854583185801589293056.000000
    <==
V3.x*1 -->  3e38 300000000000000012135895401846682943488.000000
V3.y*1 -->  3e38 300000000000000012135895401846682943488.000000
V3.z*1 -->  5e38 499999999999999969854583185801589293056.000000
    <==
V4.x*1 -->  3e38 300000000000000012135895401846682943488.000000
V4.y*1 -->  3e38 300000000000000012135895401846682943488.000000
V4.z*1 -->  3e38 300000000000000012135895401846682943488.000000
V4.t*1 -->  5e38 499999999999999969854583185801589293056.000000
    <==
V5.x*1 -->  3e38 300000000549775575777803994281145270272.000000
V5.y*1 -->  3e38 300000000549775575777803994281145270272.000000
V5.z*1 -->  3e38 300000000549775575777803994281145270272.000000
V5.t*1 -->  3e38 300000000549775575777803994281145270272.000000
V5.transmit*1 -->  5e38       inf
    <==

The current POV-Ray offerings never let you know about that last 
internal bad infinite value due the double to single type conversion 
going from the parser double specification to the internal 5d color 
float vector. Further, that bad color vect, float, 'HUGE_VAL' gets 
carried into downstream calculations, which are nearly all at double, no 
matter whether the original parse value would have been a valid double 
or not. At run time propagating that HUGE_VAL is all that can be done on 
the float to double conversion.


My updated POV_PARSER_SDL_DEBUG version of povr dies with:

---
V2.x*1 -->  3e38 300000000000000012135895401846682943488.000000
V2.y*1 -->  5e38 499999999999999969854583185801589293056.000000
    <==
V3.x*1 -->  3e38 300000000000000012135895401846682943488.000000
V3.y*1 -->  3e38 300000000000000012135895401846682943488.000000
V3.z*1 -->  3e38 499999999999999969854583185801589293056.000000
    <==
V4.x*1 -->  3e38 300000000000000012135895401846682943488.000000
V4.y*1 -->  3e38 300000000000000012135895401846682943488.000000
V4.z*1 -->  3e38 300000000000000012135895401846682943488.000000
V4.t*1 -->  3e38 499999999999999969854583185801589293056.000000
    <==
V5.x*1 -->  3e38 300000000549775575777803994281145270272.000000
V5.y*1 -->  3e38 300000000549775575777803994281145270272.000000
V5.z*1 -->  3e38 300000000549775575777803994281145270272.000000
V5.t*1 -->  3e38 300000000549775575777803994281145270272.000000
File 'vtests.inc' line 63:
Parse Error:
Value or return from vector expression.
Term 1 is too large.

The new checking sees the double HUGE_VAl value and dies.

--- On color vector components being only floats.
When just storing colors say for image display this OK.

In calculations - especially of the kind we might be doing where color 
vectors are being used in isosurfaces say, it's mostly bad news. Note 
this more affects the resulting number of significant digits than there 
be any issue with occasional infinite(inf) or not a number(nan) results. 
It's a numerical situation bad enough I'm strongly considering a move to 
double color more or less everywhere - no matter the storage hit.

Aside: Today in the source code there are internal precise color 
calculations which calculate at double, but stored colors are all at 
floats as far I know. float -> double -> calculate -> double -> float.

Bill P.

P.S. But Bill! If color floats are a numerical concern what about all 
those other settings we store for patterns say which are at single 
floats. Well, I've long been moving most of those to doubles when I run 
across them.

Post a reply to this message

I think we should very much differentiate between:

- debugging features aimed at POV-Ray developers, to figure out where we 
developers slipped up; and

- debugging features aimed at POV-Ray end users, to figure out where 
they slipped up.


Both should be kept entirely separate.

Most notably, Should we ever want to provide end-users with alternative 
binaries that they might use to debug their scenes, we MUST stil retain 
the ability to debug either of those binaries, to diagnose issues that 
users experience with only one of those.

Post a reply to this message

On 6/13/21 11:18 AM, clipka wrote:
> I think we should very much differentiate between:
> 
> - debugging features aimed at POV-Ray developers, to figure out where we 
> developers slipped up; and
> 
> - debugging features aimed at POV-Ray end users, to figure out where 
> they slipped up.
> 
> Both should be kept entirely separate.
> 
> Most notably, Should we ever want to provide end-users with alternative 
> binaries that they might use to debug their scenes, we MUST stil retain 
> the ability to debug either of those binaries, to diagnose issues that 
> users experience with only one of those.

I agree with the aim. As I implemented the POV_PARSER_SDL_DEBUG it can 
be used alone, with POV_DEBUG by default (because I think it will help 
developers and pseudo developers like me) - but the additional parser 
checking can be excluded for POV_DEBUG builds by setting 
POV_PARSER_SDL_DEBUG to 0 for a POV_DEBUG build.

---
Reality is less clean. I don't know the history, but take the 
#breakpoint parsing code enabled strictly with POV_DEBUG. It effectively 
changes the parser a little in enabling a new keyword for 'developers.'

Fresh in my mind because it's use shifts internal identifier enum values 
due being inserted as a new keyword for POV_DEBUG builds. I've been 
refining povr's new id_type() function and it's more or less the straw 
which prodded me to create a core shipped 'setidtypes.inc' include file 
when I'd intended to leave that job to users.

Also, without 'no lower case identifier checking' users can have an 
already defined 'breakpoint' in their SDL upon which the debug version 
of POV-Ray will now stop with a syntax error.

Wondering if it would be cleaner if #breakpoint was instead a function 
like pov_debug_global_counter() which was always compiled and available? 
Base it upon POV_ULONG instead of unsigned int and we'd have a counter a
step beyond what we can count with 'unsigned int' though still limited 
by the double return. Yeah, I too don't know where we'd need it, but if 
need be, it would be available already in as an SDL function. The 
feature over SDL is it would be a counter that could not be reset / 
mangled by SDL during parsing; It can only count upward and return that 
count.

Rambling: Thinking the passed argument would somehow trigger a developer 
debug counter. Maybe normally pass null/null string, but where a special 
"this_is_Christoph_Lipka" string is used ;-), it would trigger the use 
of a secondary developer 'breakpoint' counter. Don't know, thinking aloud...

Bill P.

Post a reply to this message

Am 14.06.2021 um 15:37 schrieb William F Pokorny:

> Reality is less clean. I don't know the history, but take the 
> #breakpoint parsing code enabled strictly with POV_DEBUG. It effectively 
> changes the parser a little in enabling a new keyword for 'developers.'

That feature is actually a very good example of stuff that should ALWAYS 
be enabled in developer debug versions, but NEVER in a hypothetical 
"end-user debug versions" (except for a dedicated developer debug 
version thereof).

The keyword is intended to flag lines in the scene file, and set debug 
breakpoints anywhere in POV-Ray's code that only trigger if the parser 
has already passed that line.

> Fresh in my mind because it's use shifts internal identifier enum values 
> due being inserted as a new keyword for POV_DEBUG builds. I've been 
> refining povr's new id_type() function and it's more or less the straw 
> which prodded me to create a core shipped 'setidtypes.inc' include file 
> when I'd intended to leave that job to users.

Yes, it's a very bad idea to expose any of those constants to the end 
user. The values WILL be subject to change WITHOUT notice. There are 
documented constraints to the list that would conflict with a policy of 
"keep this list frozen forever, only ever appending to the end", even if 
we were to ditch the "please keep this sorted alphabetically" policy.

It's even a very bad idea to try and catch them with hard-coded values 
in a `setidtypes.inc` to be shipped with the source, because they may 
even vary in DIFFERENT BUILDS of the SAME VERSION - even not considering 
debug versions. Most notably, future versions may include certain 
experimental features, that might require certain preprocessor switches 
to be enabled. Look at `POV_PARSER_EXPERIMENTAL_OBJ_IMPORT` for an example.

For a smarter approach, consider designing a syntax to look up the 
values by associated keyword, rather than hard coding them. Or by 
probing constructs of known type.
(If you're already doing that, forget I said anything. I haven't found 
the time & energy to have any closer look at your implementation of that 
feature.)

Also, it would be smart to introduce a separate dedicated datatype for 
type IDs, to not even tempt users to try comparing them with numeric 
literals. I'll eventually do a similar thing for stuff like RNGs and 
file IDs, which are numerical internally, but make no sense exposing to 
the user.


But I think I'm digressing here from the original topic.

Post a reply to this message