Warning: non-newbies will face palm.  Three times.

In my earliest POVing days, I had scene files on two different 
computers, transporting them back and forth via floppy disks.  Nowadays, 
both sets of files are on one computer, in separate directory trees. 
Over time, I have been gradually syncing the files and merging them into 
one tree.

The biggest challenges have been two #include files.  One of them is my 
3rd generation prefab render rig, and the other is a set of common 
definitions that I used to use a lot.  These two files are not a concern 
for me nowadays because I no longer use them for new scenes; my render 
rig is in its 4th generation, and most of the common definitions have 
found their way into the Object Collection.  But over the years, I had 
made scene-breaking changes to these files, and for the first few months 
(face palm warning) I did not document these changes.

This means that, as I merge old scene files that #include these two 
files, I need to re-render them to see whether they still work, and with 
which #include file versions.  Usually, it's just a matter of setting 
version switches, but lately, I've been merging some of my oldest 
scenes, deep into undocumented updates territory.  I am running into 
multiple levels of intractable incompatibilities.

But this latest snag--it's not even a matter of documentation, as I did 
comment the reason for the change.  I did not provide for backward 
compatibility, because I figured the change was transparent.  I 
originally defined functions Sqr(), Cubed(), and Sign().[1]  Then (two 
face palms incoming) I decided these functions were too slow, so I 
converted them to macros!

The POV-Ray 3.5 reference manual made clear that calling external macros 
incurs overhead.  Based on the date stamps of the scene files that call 
Sqr(), I had already been using POV-Ray for nearly 3 months, plenty of 
time to have learned this.  Where was my head?

And in isosurfaces?  The macros would resolve at parse time, and the 
render halt complaining about an undeclared identifier.  What possessed 
me to write general-purpose macros for use in functions?

All I can think of today is that at the time I figured these functions 
were simple enough that resolving them at parse time would not cause 
complications.  The macro overhead would be irrelevant during the render 
phase.  I must have gotten lucky with the first few scenes I rendered 
with the change.  Either that or I had been so frustrated with the 
slowness of previous isosurface renders that I never bothered 
re-rendering them after changing the #include file.  Anyhow, I pretty 
quickly stopped using these macros in new scenes.  But as I re-render 
the old scenes, the problem surfaces.

In the intervening years, I have decided that if a procedure can be 
defined as either a macro or a function, it is better to define it as a 
function.  One just has to keep in mind POV-Ray's scope leakage 
problems[2] when choosing names:

   - Once a macro is defined, its name cannot be reused as a global or
     local identifier--though it can be reused as a macro formal
     parameter.
   - Once a function is defined, its name cannot be reused as a global
     or local identifier--though it can be reused as a macro formal
     parameter.
   - Once *any* identifier is declared, it cannot be reused as a macro
     name.
   - Once *any* identifier is declared, it cannot be reused as a
     function name.
   - Once *any* identifier is declared, it cannot be reused as a
     function formal parameter.  (If you were unlucky, math.inc 3.5
     would bomb out for this very reason.  Precautions were taken
     starting with math.inc 3.6; for an example, you can take a look
     at function adj_range2().)

Also, document everything!

________________________
[1] I hadn't yet discovered f_sqr() and sgn() from math.inc, but they 
are tangential to the matter anyway.  Fortunately, the existence of 
these functions eases the conversion of the affected scenes.
[2] If you have formal computer science education, you are mortified by 
these restrictions, wondering how people smart enough to create POV-Ray 
could have allowed such boneheaded violations of modular practice.  I am 
assured that the current development team is aware of such problems. 
Yes, I will keep complaining about the scope leakage until it is fixed.

Post a reply to this message

Cousin Ricky <ric### [at] yahoocom> wrote:
> One just has to keep in mind POV-Ray's scope leakage
> problems[2] when choosing names:
>
>    - Once a macro is defined, its name cannot be reused as a global or
>      local identifier--though it can be reused as a macro formal
>      parameter.
>    - Once a function is defined, its name cannot be reused as a global
>      or local identifier--though it can be reused as a macro formal
>      parameter.
>    - Once *any* identifier is declared, it cannot be reused as a macro
>      name.
>    - Once *any* identifier is declared, it cannot be reused as a
>      function name.
>    - Once *any* identifier is declared, it cannot be reused as a
>      function formal parameter.  (If you were unlucky, math.inc 3.5
>      would bomb out for this very reason.  Precautions were taken
>      starting with math.inc 3.6; for an example, you can take a look
>      at function adj_range2().)

These things are good to know.
Scope can get pretty tricky and anxiety-inducing when I'm using things like
nested loops inside of macros, etc.
I'm never really sure when my identifier is valid/retains its value and when it
gets "released".
One day, when i get other things out of my hair, I'll probably try to write an
example scene that does all sorts of identifier backflips and other other
calisthenics.


> [1] I hadn't yet discovered f_sqr() and sgn() from math.inc,

I'm not sure why things like sgn() aren't in the source code.  It drives me
crazy when certain things I'd consider central to the operation of a lot of code
aren't written in the source code, and need to rely on include files...

And we don't really have very many proper matrix handling macros - which I kind
of found surprising for a 3D graphics package.

Thanks for you summary of the scope issues - these should make their way into
the official documentation.

Post a reply to this message

hi,

Cousin Ricky <ric### [at] yahoocom> wrote:
> Warning: non-newbies will face palm.  Three times.

:-)

> ...  One just has to keep in mind POV-Ray's scope leakage
> problems[2] when choosing names:
> ...
>    - Once *any* identifier is declared, it cannot be reused as a macro
>      name.

maybe I misunderstand.  the following works on alpha.10064268.unofficial

-----<snip>-----
#version 3.8;

global_settings {assumed_gamma 1}

#declare A = 0;

#undef A

#macro A(b_)
  b_
#end

#declare B = A(1);

#debug concat("B = ",str(B,0,0),"\n")
-----<snip>-----

(if you remove the '#undef', ie try to create macro with same name while the
identifier still _is_in_use_, error "expected undeclared identifier" occurs.
which is as it should be, aiui)

regards, jr.

Post a reply to this message

On 6/22/20 4:18 PM, Cousin Ricky wrote:
> Warning: non-newbies will face palm.  Three times.
> 

I think you're being to hard on yourself. Pretty much anytime I go back 
to something after a while, I see issues with what I did previously! :-)

POV-Ray doesn't have name spaces which would be a help with name 
collisions.

That said, the major issue I see is not having a 100% reliable way to
avoid a declare in an include causing problems for a function parameter
in another include or scene. Namely something like:

//---
// #declare _a = 1.0; // Causes issue for Fn*s

#declare Fn00 = function (_a,_b) { _a*_b }
#declare Fn01 = function (_a,_b) { _a+_b }

#declare _a = 1.0;  // This OK
#macro Mc00 (_a,_b) (_a*_b) #end
#macro Mc01 (_a,_b) (_a+_b) #end

#debug concat("Fn00(3,3) = ",str(Fn00(3,3),2,3)," () \n")
#debug concat("Fn01(3,3) = ",str(Fn01(3,3),2,3)," () \n")
#debug concat("Mc00(3,3) = ",str(Mc00(3,3),2,3)," () \n")
#debug concat("Mc01(3,3) = ",str(Mc01(3,3),2,3)," () \n")

#error "\nStop before render."
//---

We have the convention today users should not declare IDs with lower 
case characters because such things might become, or might already be, 
SDL keywords. What I'm thinking about for povr is adding checking such 
that users cannot declare/local a name with only (lower case ascii, _) 
characters. #declare _a =... would be a parsing error.

With this in place, I think it would then be the case we could always 
use something like _<lowercase_parm> for all our parameters without 
worry of collisions. The _ leading character would never be part of a 
keyword.

Am I missing something with this planned approach?

Your post touches on other changes I've made in povr. You mentioned 
math.inc defining - via our scene description language - functions
which look like fully inbuilt keywords/functions. We have also the all
lower case eval_pigment macro. I think these lower case names bad form
and I've changed them to have upper case characters in addition to 
moving the functions from math.inc to functions.inc. Plus eval_pigment I 
moved to math.inc, as Eval_Pigment, because it's a macro not a function.

Using your example, instead of the following in math.inc :

// Adjusts values in a specified range [Rmn, Rmx] to the specified
// range [Min, Max]
#declare adj_range2 =
   function (x, y, z, _Math_INC_OMn, _Math_INC_OMx) {
  ((x - y)/(z - y))*(_Math_INC_OMx - _Math_INC_OMn) + _Math_INC_OMn
   }

I have in functions.inc :

// Adjusts passed (x) values relative to a specified range [Rmn, Rmx]
// to a new value relative the specified range [Min, Max]. Normally (x) 
// in input range, but it need not be.
#declare F_adj_range2 = function (x,y,z,_mn,_mx) {
   ((x-y)/(z-y))*(_mx-_mn)+_mn
}

I want to get to where f_ prefixed functions indicate inbuilt functions. 
The F_ prefixed functions indicate SDL defined functions shipped in povr 
include files.

Something I've stumbled over is seeing odd(...) in a scene file and 
thinking it's an SDL keyword - but then not seeing any documentation for 
it. I might not always be getting today's math.inc odd() in any case. At 
least if I see the F_ prefix I'll know it's something defined in an
include and I need to be careful.

Bill P.

Post a reply to this message

Le 2020-06-23 à 04:42, jr a écrit :
> hi,
> 
> Cousin Ricky <ric### [at] yahoocom> wrote:
>> Warning: non-newbies will face palm.  Three times.
> 
> :-)
> 
>> ...  One just has to keep in mind POV-Ray's scope leakage
>> problems[2] when choosing names:
>> ...
>>     - Once *any* identifier is declared, it cannot be reused as a macro
>>       name.
> 
> maybe I misunderstand.  the following works on alpha.10064268.unofficial
> 
> -----<snip>-----
> #version 3.8;
> 
> global_settings {assumed_gamma 1}
> 
> #declare A = 0;
> 
> #undef A
> 
> #macro A(b_)
>    b_
> #end
> 
> #declare B = A(1);
> 
> #debug concat("B = ",str(B,0,0),"\n")
> -----<snip>-----
> 
> (if you remove the '#undef', ie try to create macro with same name while the
> identifier still _is_in_use_, error "expected undeclared identifier" occurs.
> which is as it should be, aiui)
> 
> regards, jr.
> 
> 

Well, that's precisely the purpose of #undef.
After the #undef statement, you are in a state similar to that of that 
identifier never having even existed.

Post a reply to this message

On 2020-06-22 5:00 PM (-4), Bald Eagle wrote:
> 
> Thanks for you summary of the scope issues - these should make their way into
> the official documentation.

You're welcome; and egregious design flaw though they be, yes, these 
booby traps should be documented.

Post a reply to this message

On 2020-06-23 4:42 AM (-4), jr wrote:
> Cousin Ricky <ric### [at] yahoocom> wrote:
>> ...
>>     - Once *any* identifier is declared, it cannot be reused as a macro
>>       name.
> 
> maybe I misunderstand.  the following works on alpha.10064268.unofficial
> 
> -----<snip>-----
> #version 3.8;
> 
> global_settings {assumed_gamma 1}
> 
> #declare A = 0;
> 
> #undef A
> 
> #macro A(b_)
>    b_
> #end
> 
> #declare B = A(1);
> 
> #debug concat("B = ",str(B,0,0),"\n")
> -----<snip>-----

Of course you can reuse any identifier after you undefine it.  But why 
should you have to go through such nonsense?

> (if you remove the '#undef', ie try to create macro with same name while the
> identifier still _is_in_use_, error "expected undeclared identifier" occurs.
> which is as it should be, aiui)

No. It. Shouldn't.

What if the macro was defined in an include file written by someone 
else? If the user declares A, they obviously have no need for macro A, 
so why should they have to worry about whether a macro by that name 
already exists?

But this is less serious an issue; the fact remains that if you declare 
a *local* variable with the same name as a macro, the scene will not 
parse.  This means that anyone who writes a macro must make sure they do 
not use local variables with the same name as a macro in any other 
include file--even include files written by total strangers.  The same 
goes for function formal parameters.

This TOTALLY 100% defeats the purpose of local variables and formal 
parameters!  In order to write a robust POV include file, you must 
essentially read the mind of every POVer on the planet.

This is why I complain about scope leakage.  As far as I'm concerned, 
the only acceptable level of scope leakage is ZERO.

Post a reply to this message

On 2020-06-23 7:28 AM (-4), William F Pokorny wrote:
> POV-Ray doesn't have name spaces which would be a help with name 
> collisions.

Yes, that certainly would help!  It could resolve the disagreement I 
have with jr. over macro name persistence.  I would love it, although it 
would be massive work for the dev team.  Fixing the local variable and 
function formal parameter problem is far more important and should be 
prioritized.

> That said, the major issue I see is not having a 100% reliable way to
> avoid a declare in an include causing problems for a function parameter
> in another include or scene. Namely something like:
> 
> [snip]
> 
> We have the convention today users should not declare IDs with lower 
> case characters because such things might become, or might already be, 
> SDL keywords. What I'm thinking about for povr is adding checking such 
> that users cannot declare/local a name with only (lower case ascii, _) 
> characters. #declare _a =... would be a parsing error.
> 
> With this in place, I think it would then be the case we could always 
> use something like _<lowercase_parm> for all our parameters without 
> worry of collisions. The _ leading character would never be part of a 
> keyword.
> 
> Am I missing something with this planned approach?

I suppose it could work.  It could potentially break old scenes, but I 
don't suppose many such scenes exist.  None of the stock include files 
have any such names that I can find.

How widespread are you recommending this convention?  It would violate 
the existing Object Collection namespace conventions, unless we amend 
the conventions.  Since Chris Bartlett disappeared, I don't know if we 
even have a formal mechanism for deciding Object Collection policy.

> Your post touches on other changes I've made in povr. You mentioned 
> math.inc defining - via our scene description language - functions
> which look like fully inbuilt keywords/functions. We have also the all
> lower case eval_pigment macro. I think these lower case names bad form
> and I've changed them to have upper case characters in addition to 
> moving the functions from math.inc to functions.inc.

The problem with renaming functions and macros is that it would break 
older scenes.  But keeping the old names alongside the new names would 
solve that problem.  Since these old names are already part of the 
POV-Ray canon, avoiding new keywords with these names should not be a 
problem.

> Plus eval_pigment I 
> moved to math.inc, as Eval_Pigment, because it's a macro not a function.

It seems to me that Eval_Pigment() belongs in textures.inc.

> I want to get to where f_ prefixed functions indicate inbuilt functions. 
> The F_ prefixed functions indicate SDL defined functions shipped in povr 
> include files.
> 
> Something I've stumbled over is seeing odd(...) in a scene file and 
> thinking it's an SDL keyword - but then not seeing any documentation for 
> it. I might not always be getting today's math.inc odd() in any case. At 
> least if I see the F_ prefix I'll know it's something defined in an
> include and I need to be careful.

I personally use fn_ or Fn_, because I already use f_ for finishes.  Of 
course, I can always distinguish between my finish and inbuilt functions 
by noting the one or more capital letters in my finish.

What do you think of e and o in consts.inc?  I think o would be better 
as O, because the Cartesian origin is normally denoted with a capital O. 
  But Euler's constant is always lowercase e.  Would it be worth making 
a keyword?

Post a reply to this message

hi,

Cousin Ricky <ric### [at] yahoocom> wrote:
> On 2020-06-23 4:42 AM (-4), jr wrote:
> > Cousin Ricky <ric### [at] yahoocom> wrote:
> >> ...
> > maybe I misunderstand.  ...
>
> Of course you can reuse any identifier after you undefine it.  But why
> should you have to go through such nonsense?
>
> > (if you remove the '#undef', ie try to create macro with same name while the
> > identifier still _is_in_use_, error "expected undeclared identifier" occurs.
> > which is as it should be, aiui)
>
> No. It. Shouldn't.

hm, we all see things differently.  fwiw, the fact that POV-Ray is .. lax enough
to allow me to do:

#declare A = "foo";
#declare A = <1,2,3>;
#declare A = 0;

without complaint, makes me unhappy.  I'd much prefer "stronger" typing.

> What if the macro was defined in an include file written by someone
> else? If the user declares A, they obviously have no need for macro A,
> so why should they have to worry about whether a macro by that name
> already exists?

for the same reason one would be careful when including some library header in a
C translation unit?  given the absence of namespaces (as you note else-thread),
there's only careful design and documentation of .inc file, to alleviate the
situation.

need to think, and perhaps experiment, re "scope leakage".


regards, jr.

Post a reply to this message

Cousin Ricky <ric### [at] yahoocom> wrote:

> How widespread are you recommending this convention?

It might help obviate "reading the mind of every Povver" if some mechanism were
implemented to prepend/append some designation to the variables to make them
"unique" to any given include file.

BE23062020inc_var1 = pow(e, i*pi);

Verbose, more work, jr will have an apopleptic fit, but serves as a functional
workaround.

I would love to be able to declare identifiers using some form of formula or
concatenation - though I don't know what that would involve in the source code.

If I wanted a01 through a10, then I could write a for loop that would declare
those identifiers.

#for (N, 1, 10)
#declare Identifier ("a" + str(N)) = pow(N, 2);
#end

(not proper syntax, but you get the _idea_)

And if THAT were possible, then at the top of the include file, one could
#declare INC_PREFIX = "BE23062020inc_";
write a small macro to prepend that to any identifier name, and then

#declare INC(a) = 0;

would be the/a way you'd handle all of the names in that file.

I know we have that parse thing (the name always escapes me) that writes to a
file and includes it back in...

> > Your post touches on other changes I've made in povr. You mentioned
> > math.inc defining - via our scene description language - functions
> > which look like fully inbuilt keywords/functions. We have also the all
> > lower case eval_pigment macro. I think these lower case names bad form
> > and I've changed them to have upper case characters in addition to
> > moving the functions from math.inc to functions.inc.

Why not make them ... fully inbuilt keywords/functions?

> The problem with renaming functions and macros is that it would break
> older scenes.  But keeping the old names alongside the new names would
> solve that problem.  Since these old names are already part of the
> POV-Ray canon, avoiding new keywords with these names should not be a
> problem.

How much does it take to implement an alias system?
It would be ever so nice to be able to rename inbuilt keywords and functions.

> > Plus eval_pigment I
> > moved to math.inc, as Eval_Pigment, because it's a macro not a function.
>
> It seems to me that Eval_Pigment() belongs in textures.inc.

Move it into source code!  :)

> > I want to get to where f_ prefixed functions indicate inbuilt functions.
> > The F_ prefixed functions indicate SDL defined functions shipped in povr
> > include files.
> >
> > Something I've stumbled over is seeing odd(...) in a scene file and
> > thinking it's an SDL keyword - but then not seeing any documentation for
> > it. I might not always be getting today's math.inc odd() in any case. At
> > least if I see the F_ prefix I'll know it's something defined in an
> > include and I need to be careful.

odd, even, and those sorts of functions should be in source code.
I find it ridiculous to include a whole file simply for one little function.
Or have to hunt down the include file to copy/paste the code.

> What do you think of e and o in consts.inc?  I think o would be better
> as O, because the Cartesian origin is normally denoted with a capital O.
>   But Euler's constant is always lowercase e.  Would it be worth making
> a keyword?

YES

Post a reply to this message