The POV-Ray 3.x SDL is, at its core, a language to describe static
hierarchical data structures in a quite efficient manner, with a
scripting language bolted on top later to also allow for dynamic
creation of data.

I think there's nothing fundamentally wrong with this approach;
specifically, I think the data description aspect of a POV-Ray scene is
so important that it is wiser to start with a data description language
and bolt a scripting language on top of it, rather than start with a
scripting language and extend it with data description elements (the
latter is the path of evolution that led to JSON). Therefore, in this
random proposal I will follow that same path.

As a starting point, I'll take an existing language that I consider
reasonably efficient at describing static hierarchical data structures:
JSON (defined in ECMA-404). For those who don't know it, this is what an
arbitrary JSON file looks like:

    {
      "firstName": "John",
      "lastName": "Smith",
      "isAlive": true,
      "age": 25,
      "address": {
        "streetAddress": "21 2nd Street",
        "city": "New York",
        "state": "NY",
        "postalCode": "10021-3100"
      },
      "phoneNumbers": [
        {
          "type": "home",
          "number": "212 555-1234"
        },
        {
          "type": "office",
          "number": "646 555-4567"
        }
      ],
      "children": [],
      "spouse": null
    }

A simple POV-Ray scene described in this format might look like this:

    {
      "camera": {
        "up": [ 0, 1, 0 ],
        "right": [ 1.33, 0, 0 ],
        "direction": [ 0, 0, -1 ],
        "location": [ 0, 0, 5 ],
        "look_at": [ 0, 0, 0 ]
      },
      "objects": [
        {
          "object_type": "light_source",
          "location": [ -100, 100, 100 ],
          "colour": "#FFFFFF"
        },
        {
          "object_type": "sphere",
          "location": [ 0, 0, 0 ],
          "radius": 1,
          "texture": {
            "pigment": {
              "colour": "#FFFFFF"
            }
          }
        }
      ]
    }

Notice how I chose to use the HTML-style colour format; this is a
comparatively random choice, probably because JSON lives in the same
ecosystem as HTML.

The first thing that strikes me is the fact that we have a vast number
of double quotes I, as a user, wouldn't want to have to type. As we'll
devising our own language anyway, we might therefore try to avoid them,
and I think we can accomplish this (even without breaking backward
compatibility to pure JSON) by addition of the following rule:

* If a string appears on the left-hand side of a name/value pair,
contains only uppercase letters, lowercase letters, digits and/or
underscores, does not start with a digit, and is neither "true" nor
"false" nor "null" (nor equivalent to any keyword we might add later),
the enclosing double quotes are optional.

Note that this mandates that we can't use character sequences of this
form for any other purpose.

We can also simplify our colour notation in a similar manner:

* If a string starts with "#", and the remainder of the string is
comprised solely of hexadecimal digits, the enclosing double quotes are
optional.

Also, the "object_type" thing bothers me; I therefore introduce /types/
to the language:

* An /object/ (in the JSON sense) may be preceded by a string specifying
its /type/.

This leaves us with the following scene description:

    {
      camera: {
         up: [ 0, 1, 0 ],
         right: [ 1.33, 0, 0 ],
         direction: [ 0, 0, -1 ],
         location: [ 0, 0, 5 ],
         look_at: [ 0, 0, 0 ]
      },
      objects: [
        light_source {
          location: [ -100, 100, 100 ],
          colour: #FFFFFF
        },
        sphere {
          location: [ 0, 0, 0 ],
          radius: 1,
          texture: {
            pigment: {
              colour: #FFFFFF
            }
          }
        }
      ]
    }

I think this constitutes a reasonable format for static scene
description; it's about as concise as the current format.

Now for bolting on a scripting language:

First we need control structure keywords; we don't want them to collide
with our unquoted strings, so I'll borrow a feature from POV-Ray 3.x
SDL: I'll prefix them with "#". For starters, I'll use the following:

    #if(CONDITION)
      ...
    #elif(CONDITION)
      ...
    #else
      ...
    #end

    #loop NAME
      ...
      #if(CONDITION) #break NAME #end
      ...
      #continue NAME
    #end

That'll be all we really need; we can bolt on more later (though we need
to make sure we don't get collisions with colour names).

Next we need variables; again we don't want them to collide with our
unquoted strings, so I'll use just about the same trick, except that I
use "@" to refer to variables.

As a matter of fact I'll define "@" as a prefix operator for string
expressions, meaning "the content of the variable with the name". I'll
use the following syntax to define variables for now:

    #global {
      "A variable name with blanks": 4711,
      Foo: [ 0, 1, 2 ],
      Bar: "Foo"
    }
    #local { ... }

I can now use the variables as follows:

    sphere: {
      radius: @"A variable name with blanks"
      center: @@Bar // look what I just did there
    }

I'll also extend the language by introducing non-literal values,
allowing mathematical operations and other some such, but I'll not go
into details there for now.

We could now modify our scene as follows:

    {
      camera: {
        up: [ 0, 1, 0 ],
        right: [ @image_width / @image_height, 0, 0 ],
        direction: [ 0, 0, -1 ],
        location: [ 0, 0, 5 ],
        look_at: [ 0, 0, 0 ]
      },
      objects: [
        light_source {
          location: [ -100, 100, 100 ],
          colour: #FFFFFF
        },
        #local { i: 1, n: 10 }
        #loop
          sphere {
            location: @i * [ 2, 0, 0 ],
            radius: 1,
            texture: {
              pigment: {
                colour: #FFFFFF
              }
            }
          }
          #local { i: @i + 1 }
          #if (@i <= @n)
            ,
            #continue
          #else
            #break
          #end
        #end
      ]
    }

There. Behold your new SDL, chilren of POV-Ray.


Or don't. Make alternative suggestions, or proceed from here.

One possible change to the syntax could be the use of regular braces
around list items instead of square brackets to specifically denote
vectors, if only to make it more pleasant to read. (As for using angular
brackets for this purpose, I never liked that part of POV-Ray 3.x's
syntax in the first place; also, in specific cases it has the potential
to collide with the greater-than sign.)

I'd also love to add ranges to the set of types, using a syntax akin to
this:

    [ 1 .. 20 ] // range from 1 to 20, both inclusive
    ( 1 .. 20 ) // range from 1 to 20, both exclusive
    [ 1 .. 20 ) // range from 1 inclusive to 20 exclusive
    ( 1 .. 20 ] // range from 1 exclusive to 20 inclusive

In general, I'd love the syntax to be geared more towards mathematical
notation, and I'd also like it to make use of Unicode mathematical
symbols as (entirely optional) syntactic sugar.


BTW, I don't consider the above syntax really good, most notably because
the possible location of control structures is poorly defined, which
makes the whole thing unsuited for a compiled language. It's really just
one random proposal thrown together in a matter of a few hours, in hopes
to kick off a more productive discussion about how the next generation
SDL might look like.

Post a reply to this message

On 12/12/2015 9:42 AM, clipka wrote:
> The POV-Ray 3.x SDL is, at its core, a language to describe static
> hierarchical data structures in a quite efficient manner, with a
> scripting language bolted on top later to also allow for dynamic
> creation of data.
>
> I think there's nothing fundamentally wrong with this approach;
> specifically, I think the data description aspect of a POV-Ray scene is
> so important that it is wiser to start with a data description language
> and bolt a scripting language on top of it, rather than start with a
> scripting language and extend it with data description elements (the
> latter is the path of evolution that led to JSON). Therefore, in this
> random proposal I will follow that same path.

I like JavaScript, so JSON seems reasonable.

One quick note: you shouldn't use hex for colors because then you can't 
perform precise calculations. Better to use float like we do now.


Mike

Post a reply to this message

On 12/13/2015 3:23 AM, Mike Horvath wrote:
> I like JavaScript, so JSON seems reasonable.
>
> One quick note: you shouldn't use hex for colors because then you can't
> perform precise calculations. Better to use float like we do now.
>
>
> Mike


In fact, I would rather just use JavaScript for everything.


Mike

Post a reply to this message

Mike Horvath <mik### [at] gmailcom> wrote:
> One quick note: you shouldn't use hex for colors because then you can't
> perform precise calculations. Better to use float like we do now.

How about just translating it to the standard 5-D float vector at parse time, as
is done with the srgb keyword series in POV-Ray 3.7?

Post a reply to this message

Am 13.12.2015 um 16:43 schrieb Cousin Ricky:
> Mike Horvath <mik### [at] gmailcom> wrote:
>> One quick note: you shouldn't use hex for colors because then you can't
>> perform precise calculations. Better to use float like we do now.
> 
> How about just translating it to the standard 5-D float vector at parse time, as
> is done with the srgb keyword series in POV-Ray 3.7?

My idea would be to use HTML colour format for colours in sRGB space,
and some other syntax to specify linear colours; internally, they'd both
be translated to a vector-like linear format, and math operations could
be done on them.

Colours would /not/ be treated as strings internally.


As for translating colours to "standard 5-D float vectors", that won't
happen:

(1) Vectors are vectors, and colours are colours. In my book, using one
and the same type for them is one of the biggest blunders in POV-Ray's
current SDL.

(2) The RGBFT format is another one of the biggest blunders in the
history of POV-Ray, and unfortunately it's more deeply entrenched in the
rendering engine. Wherever any transparency is involved, internal math
should use two RGB values, one representing the opaque component and
another the transmissive one.

Post a reply to this message

On 12/12/2015 09:42 AM, clipka wrote:
 >

 From a users perspective I think the tendency is to wish for an SDL 
language close to the programming languages we currently use as learning 
new stuff to the point of being quick with it takes a lot of time. Much 
of my own povray use over time has come from hacks in other languages to 
generate existing SDL for whatever job - using TCL mostly of late - 
though I write native POV SDL too.

A good many software libraries tools provide and API in C/C++ (or create 
one with SWIG I've read) to enable a set of bindings from a tool to 
other languages of the day. I've never heard mention of such an API with 
POV-Ray.

I'd first recommend the developers continue whatever code restructuring 
is in flight to enable a clean API so bindings to language xyz, or 
running POV-Ray as a compiled C++ program as Jérôme suggested, is 
possible. Perhaps with a defined API or an API more easily determined 
automatically from a cleaner code base, bindings for various languages 
will then just spring up. ;-) Even if not, it seems like we'd be better 
set up to create any new POV-Ray SDL at such a time.

I do think a POV-Ray specific SDL is a good idea as probably that SDL 
will always be the only completely enabled language interface. Further, 
it permits a standard interchange language for methods and techniques 
among users.

I generally like your random SDL proposal, but expect the difficulties 
will be in the details of any new SDL implementation.

Is the eventual aim also to enable SMP during parsing?

Other comments in line below.

Bill P.

> The POV-Ray 3.x SDL is, at its core, a language to describe static
> hierarchical data structures in a quite efficient manner, with a
> scripting language bolted on top later to also allow for dynamic
> creation of data.
>
> I think there's nothing fundamentally wrong with this approach;
> specifically, I think the data description aspect of a POV-Ray scene is
> so important that it is wiser to start with a data description language
> and bolt a scripting language on top of it, rather than start with a
> scripting language and extend it with data description elements (the
> latter is the path of evolution that led to JSON). Therefore, in this
> random proposal I will follow that same path.
>
.......
>
> This leaves us with the following scene description:
>
>      {
>        camera: {
>           up: [ 0, 1, 0 ],
>           right: [ 1.33, 0, 0 ],
>           direction: [ 0, 0, -1 ],
>           location: [ 0, 0, 5 ],
>           look_at: [ 0, 0, 0 ]
>        },
>        objects: [
>          light_source {
>            location: [ -100, 100, 100 ],
>            colour: #FFFFFF
>          },
>          sphere {
>            location: [ 0, 0, 0 ],
>            radius: 1,
>            texture: {
>              pigment: {
>                colour: #FFFFFF
>              }
>            }
>          }
>        ]
>      }

Here I am not getting the real purpose for objects: [] other than to 
enable us to drop the colons for light_source and sphere - something I 
think less clear than light_source: {} and sphere: {}.

.......
>
> As a matter of fact I'll define "@" as a prefix operator for string
> expressions, meaning "the content of the variable with the name". I'll
> use the following syntax to define variables for now:
>
>      #global {
>        "A variable name with blanks": 4711,
>        Foo: [ 0, 1, 2 ],
>        Bar: "Foo"
>      }
>      #local { ... }
>
> I can now use the variables as follows:
>
>      sphere: {
>        radius: @"A variable name with blanks"
>        center: @@Bar // look what I just did there
>      }

I like the idea of #global{} and #local{} blocks for defining variables. 
I wonder about @@+ when the references are not 1 to 1, or perhaps they 
must be? Thinking, for example, about three deep and Foo: ["MyX",...]

I'm also a little fuzzy on how we would define and reference predefined 
entities later by name. Would it be something like:

{
   camera00: {
     camera: {
       ...
     }
   }
   ...
   camera: { @camera00 ... }
}

?

.....

>
> There. Behold your new SDL, chilren of POV-Ray.
>
>

:-) Cool, I'm young again.

Post a reply to this message

Am 13.12.2015 um 20:29 schrieb William F Pokorny:

> From a users perspective I think the tendency is to wish for an SDL
> language close to the programming languages we currently use as learning
> new stuff to the point of being quick with it takes a lot of time. Much
> of my own povray use over time has come from hacks in other languages to
> generate existing SDL for whatever job - using TCL mostly of late -
> though I write native POV SDL too.

Obviously catering to this wish isn't possible for all the user base,
given how different the languages are they use. You mention Tcl, IIRC
others have mentioned python, then of course there's C and the vast
number of its offspring like C++, Java, C#, the recently mentioned
JavaScript etc., others may prefer Perl, maybe Lua -- you certainly get
the point.

Thus, as long as you think of POV-Ray as an application that reads an
input file and generates an image from it, what you wish for is not
going to happen for at least a portion of the user base -- unless of
course the new SDL will be very similar to the current one.


But it seems that your idea is heading in a different direction:
Providing POV-Ray not as an application, but as a library.

> A good many software libraries tools provide and API in C/C++ (or create
> one with SWIG I've read) to enable a set of bindings from a tool to
> other languages of the day. I've never heard mention of such an API with
> POV-Ray.

It has been mentioned as an idea, and a goal to aim for. That's a start.

> I'd first recommend the developers continue whatever code restructuring
> is in flight to enable a clean API so bindings to language xyz, or
> running POV-Ray as a compiled C++ program as Jérôme suggested, is
> possible. Perhaps with a defined API or an API more easily determined
> automatically from a cleaner code base, bindings for various languages
> will then just spring up. ;-) Even if not, it seems like we'd be better
> set up to create any new POV-Ray SDL at such a time.

I think it won't hurt to do some brainstorming now already: Defining the
new language might be a lengthy process, and if we start that process
only when the API is actually established, I think we're losing time
unnecessarily.

For the sake of designing an SDL, actually we need to know little of the
ultimate API: We need to know that there'll be a scene comprised of a
hierarchy of elements, each element having particular properties
depending on the type of element, with some of the properties being
ordered lists of elements. That's about it.

The current SDL has all those elements and properties hard-wired, but
that's far from necessary; to the contrary, it only creates a bloated
language and parser, with a lot of potential for inconsistencies.


> Is the eventual aim also to enable SMP during parsing?

I've thought about this one, but I'm not sure whether it is a good idea.
SMP comes with plenty of pitfalls, and makes the virtual machine and its
integration into the application more difficult.

Maybe at a later stage.

>>
>>      {
>>        camera: {
>>           up: [ 0, 1, 0 ],
>>           right: [ 1.33, 0, 0 ],
>>           direction: [ 0, 0, -1 ],
>>           location: [ 0, 0, 5 ],
>>           look_at: [ 0, 0, 0 ]
>>        },
>>        objects: [
>>          light_source {
>>            location: [ -100, 100, 100 ],
>>            colour: #FFFFFF
>>          },
>>          sphere {
>>            location: [ 0, 0, 0 ],
>>            radius: 1,
>>            texture: {
>>              pigment: {
>>                colour: #FFFFFF
>>              }
>>            }
>>          }
>>        ]
>>      }
> 
> Here I am not getting the real purpose for objects: [] other than to
> enable us to drop the colons for light_source and sphere - something I
> think less clear than light_source: {} and sphere: {}.

It's just a vehicle to shoehorn the data structure into what JSON can
accomodate.

If I used the notation

    scene: {
      light_source: { ... }
      sphere: { ... }
    }

it would mean that the scene will have one property named "sphere"
containing at most one sphere, another property named "light_source"
containing at most one light_source, and -- by extension -- one property
named "box" containing at most one box, one property named
"height_field" containing at most one height field, etc.

That's obviously not what we want.


This is actually one of the things that's been giving me the biggest
headaches: I'd prefer to have virtually the same syntax for specifying
the objects in a scene, and for specifying the texture of an object; but
while in the former case I actually want to build a multi-element list,
in the latter case I want to set a single-element property.

Thus, I can't let the language just define arbitrary data hierarchies
(like JSON), and translate those to actual scene elements in a
subsequent steps; instead, the process of defining the data hierarchies
must have some knowledge of the data elements being defined.

>>      #global {
>>        "A variable name with blanks": 4711,
>>        Foo: [ 0, 1, 2 ],
>>        Bar: "Foo"
>>      }
>>      #local { ... }
>>
>> I can now use the variables as follows:
>>
>>      sphere: {
>>        radius: @"A variable name with blanks"
>>        center: @@Bar // look what I just did there
>>      }
> 
> I like the idea of #global{} and #local{} blocks for defining variables.

Isn't that neat?
It just came naturally when I thought about how, in this particular
language, the exact syntax of a "#declare" statement (which I later
decided should be "#global") should be, and I think I might carry over
that idea to other proposals.

> I wonder about @@+ when the references are not 1 to 1, or perhaps they
> must be? Thinking, for example, about three deep and Foo: ["MyX",...]

I'm not exactly sure I understand what you mean there.
If the "@" operator is applied to something that isn't a string, it bombs.

> I'm also a little fuzzy on how we would define and reference predefined
> entities later by name. Would it be something like:
> 
> {
>   camera00: {
>     camera: {
>       ...
>     }
>   }
>   ...
>   camera: { @camera00 ... }
> }

Like in the current SDL, we'd use global or local variables, like so:

  #declare { camera00: camera { ... } }
  ...
  camera: @camera00

Post a reply to this message

>          #local { i: 1, n: 10 }
>          #loop

...

>            #local { i: @i + 1 }
>            #if (@i <= @n)
>              ,
>              #continue
>            #else
>              #break
>            #end
>          #end

You know something has gone very wrong when a for loop looks like that :-)

How about:

for i 1 10
{
...
}

Post a reply to this message

Am 14.12.2015 um 12:55 schrieb scott:
>>          #local { i: 1, n: 10 }
>>          #loop
> 
> ....
> 
>>            #local { i: @i + 1 }
>>            #if (@i <= @n)
>>              ,
>>              #continue
>>            #else
>>              #break
>>            #end
>>          #end
> 
> You know something has gone very wrong when a for loop looks like that :-)

No, it just means that the syntax is a bit on the minimalistic side ;)

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:

>
> My idea would be to use HTML colour format for colours in sRGB space,
> and some other syntax to specify linear colours; internally, they'd both
> be translated to a vector-like linear format, and math operations could
> be done on them.

Personally, as someone who just likes to make nice-looking images and animations
(and not being a 'programmer' per se), the use of HTML color formatting seems
*much* less intuitive than our current vector format. For example, <1,.4,.7> vs.
something like FFDFB  (I made up that last term!) With the vector approach, I
can write a 'first approximation' color quite easily if I want to, then tweak it
by eye. The hex(?) way of doing it requires either a lookup table, or else a
GOOD intuitive knowledge of that kind of coding approach and its visual results.
I suppose there are crack programmers who think in such terms, but I'm certainly
not one of 'em. ;-) (Long ago, I got into HTML coding-- before style sheets--
and even then I thought it 'odd' to have to specify colors in what seemed to be
such an arcane manner.)
>
> As for translating colours to "standard 5-D float vectors", that won't
> happen:
>
> (1) Vectors are vectors, and colours are colours. In my book, using one
> and the same type for them is one of the biggest blunders in POV-Ray's
> current SDL.
>

Here's a funny thing: I've always been under the impression that the original
developers actually *wanted* vectors and colors to use the same coding scheme,
explicitly. For ease of understanding and computation in the SDL. I.e., "A
vector is a color is a...." From a conceptual standpoint, such an arrangement
does make sense, to me.

Another question arises concerning dot-notation (i.e., pulling one of the color
components out of the color vector, for use.) I don't see how that would work
using the FFDFB approach. (I suppose it *could* simply be specified as
FFDFD.red, for example, with POV-ray's internal engine doing the hex-to-float
conversion invisibly. But again, that doesn't seem very intuitive from a user's
standpoint.)

I hope I haven't gotten into meaningless or extraneous details re: your
discussion.

Post a reply to this message

Op 2015-12-12 om 15:42 schreef clipka:
> The POV-Ray 3.x SDL is, at its core, a language to describe static
> hierarchical data structures in a quite efficient manner, with a
> scripting language bolted on top later to also allow for dynamic
> creation of data.

How much of a "language" does POV-Ray actually need? How bare-bone can a 
human readable fast parsing textual interface be? For example, when I 
craete scene's in Python I enroll all loops so POV-Ray has not to do it 
and the parsing speeds up quite a lot.

Could it work, a bare-bone SDL and a pre-processor in a good scripting 
language, so all libraries of that are also usable?

(the pre-processor could then be made to also output other scene formats 
OpenGL, WebGL etc)

Ingo

Post a reply to this message

> Personally, as someone who just likes to make nice-looking images and animations
> (and not being a 'programmer' per se), the use of HTML color formatting seems
> *much* less intuitive than our current vector format. For example, <1,.4,.7> vs.
> something like FFDFB  (I made up that last term!) With the vector approach, I
> can write a 'first approximation' color quite easily if I want to, then tweak it
> by eye.

I think clipka's point is that you would use the vector format with the 
linear colour identifier, and the hex format with the sRGB identifier. 
That makes perfect sense to me, because if you're tinkering with 
something like <1,.4,.7> you probably want that to be a linear colour 
(ie green is 40% of maximum brightness) - unless you can do the sRGB 
conversions in your head :-)

Post a reply to this message

"Kenneth" <kdw### [at] gmailcom> wrote:

> I hope I haven't gotten into meaningless or extraneous details re: your
> discussion.

Not at all. I consider it valuable feedback.

As for "a colour is a vector", one thing that bothers me very much about this is
that, although it matches the typical representation in computer graphics, the
physics of colours is quite different, and performing colour maths in a
3-dimensional coordinate space is just a very poor approximation of physical
reality. The relevant buzzword is "spectral rendering".

Making the colour type intimately related to the vector type ties down POV-Ray's
colour math to the RGB world, and hinders the development of modified versions
that support spectral rendering.

Post a reply to this message

ingo <ing### [at] gmailcom> wrote:
> Op 2015-12-12 om 15:42 schreef clipka:
> > The POV-Ray 3.x SDL is, at its core, a language to describe static
> > hierarchical data structures in a quite efficient manner, with a
> > scripting language bolted on top later to also allow for dynamic
> > creation of data.
>
> How much of a "language" does POV-Ray actually need? How bare-bone can a
> human readable fast parsing textual interface be? For example, when I
> craete scene's in Python I enroll all loops so POV-Ray has not to do it
> and the parsing speeds up quite a lot.
>
> Could it work, a bare-bone SDL and a pre-processor in a good scripting
> language, so all libraries of that are also usable?
>
> (the pre-processor could then be made to also output other scene formats
> OpenGL, WebGL etc)

In a distributed rendering environment, it may be preferable to avoid
"unrolling" complex scenes to reduce network load.

Also, true pre-processor languages are prone to result in an overall syntax that
is difficult to analyze at edit-time.

Post a reply to this message

So... after 30+ year ago, here is the answer tomy question: when is 
POV-Ray 4 is going to go out? Me and many others, that were quickly met 
with belligerent: it's going to be when is done, now STFU about it and 
never ask the fucking question again in your life! Or be banned from the 
entire pov-ray newsgroup! Or something to that shock effect; but I feel 
other and I contributed my 2 cents to keep things, at least alive, if 
not greasing them a little, until they started going; hey, I pushed Warp 
to join as an active programmer, so your welcome bitches! LOL :-D

Post a reply to this message

So... after 30+ year ago, here is the answer tomy question: when is 
POV-Ray 4 is going to go out? Me and many others, that were quickly met 
with belligerent: it's going to be when is done, now STFU about it and 
never ask the fucking question again in your life! Or be banned from the 
entire pov-ray newsgroup! Or something to that shock effect; but I feel 
other and I contributed my 2 cents to keep things, at least alive, if 
not greasing them a little, until they started going; hey, I pushed Warp 
to join as an active programmer, so your welcome bitches! LOL :-D

Post a reply to this message

So... after 30+ years

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> As a starting point, I'll take an existing language

Okay, Maybe what follows is incompatible with this language I know nothing
about, so ignore my remarks in that eventuality because I do value that idea of
leveraging the best and closest standard if possible.


> A simple POV-Ray scene described in this format might look like this:
[...]
>     {
>       camera: {
>          up: [ 0, 1, 0 ],
>          right: [ 1.33, 0, 0 ],
>          direction: [ 0, 0, -1 ],
>          location: [ 0, 0, 5 ],
>          look_at: [ 0, 0, 0 ]
>       },
>       objects: [
>         light_source {
>           location: [ -100, 100, 100 ],
>           colour: #FFFFFF
>         },
>         sphere {
>           location: [ 0, 0, 0 ],
>           radius: 1,
>           texture: {
>             pigment: {
>               colour: #FFFFFF
>             }
>           }
>         }
>       ]
>     }


The  first thing bothering me is the combination of colons and brace characters
in some places or maybe rather that it seems it can't be used the same  way
elsewhere?
Would we have to write camera: {
It would seem a little cluttered to have to use a : for just specifying the
opening of a block. do we have to ? ... why not:

       camera{
          up: [ 0, 1, 0 ],
          right: [ 1.33, 0, 0 ],
          direction: [ 0, 0, -1 ],
          location: [ 0, 0, 5 ],
          look_at: [ 0, 0, 0 ]
       },

Also for same reason, would the comma after the brace really be necessary ?



> Next we need variables; again we don't want them to collide with our
> unquoted strings, so I'll use just about the same trick, except that I
> use "@" to refer to variables.

Fine,

>       center: @@Bar // look what I just did there

I'm not sure I understand, maybe that's why, but it looks a little barbarian to
my delicate profane eye :-) Does json force to specify like this instead of some
"object" dot lookup operator?

> There. Behold your new SDL, chilren of POV-Ray.

Yay ! great to see some direction, and a great start !

> One possible change to the syntax could be the use of regular braces
> around list items instead of square brackets to specifically denote
> vectors, if only to make it more pleasant to read.

I believe readability would suffer and prefer your proposition versus curly
braces everywhere whith no rythmic to distinguish what's inside what though the
too-many-different-enclosing-signs margin is probably easily reached, so <>
could go away as far as i'm concerned

> I'd also love to add ranges to the set of types, using a syntax akin to
> this:
>
>     [ 1 .. 20 ] // range from 1 to 20, both inclusive
>     ( 1 .. 20 ) // range from 1 to 20, both exclusive
>     [ 1 .. 20 ) // range from 1 inclusive to 20 exclusive
>     ( 1 .. 20 ] // range from 1 exclusive to 20 inclusive

Personally I think this looks pretty cool! But again I'm no expert at all.

> symbols as (entirely optional) syntactic sugar.

Sounds awesome as well and a breath of fresh air for such a feature rich
program, it also goes well with the cypher obscure reputation of POV, except for
once it would have some real benefits, I assume, in code parsing computation
times ?  But with the prerequisites that they should have an explicit
alternative for when we don't know or have any internet to check for the code to
type. most people never enter a unicode special number their whole life. but
maybe the parsing times could really be worth that learning?

This brings to my mind the topic even outside of the mathematical field of some
unicode symbols or even some more common non ascii characters, sorry if too off
topic for lack of terminology and specialists concepts here, but my naive
expectations for POV sdl of my dreams :

a)It should strive to still look intuitive and elegant to non programmers,
preferably one & looks better than two or three && or @@  except where
conventions are long there in several mainstream languages like comments with
doubled chars // for many languages and some alternative doubled sign in many
more others.

b)May we try to keep it easy to type with less modifier keys across the two or
three most used keyboard layouts, since we don't want povers to have carpal
tunnel. (but probably the most cryptic unicode are easier to call by their id
number than some of the characters available of the keyboard layouts requiring
to type a space before they appear and one of two alt keys etc...

I'm getting lost in that stream of thoughts, wishes and good intentions, so I
should stop especially as I trust the experts around here a lot more !

Thanks for giving this a lot of thought!

Post a reply to this message

delete this message, I can't do it myself from Mozilla Thunderbird, 
tried to unsubscribe and subscribe several times, tried several methods

Post a reply to this message

delete this message, I can't do it myself from Mozilla Thunderbird, 
tried to unsubscribe and subscribe several times, tried several methods, 
it was a moment of insanity, sorry

Post a reply to this message

delete this message, I can't do it myself from Mozilla Thunderbird, 
tried to unsubscribe and subscribe several times, tried several methods, 
it was a moment of insanity, sorry

Post a reply to this message

"Mr" <nomail@nomail> wrote:
> clipka <ano### [at] anonymousorg> wrote:

> > A simple POV-Ray scene described in this format might look like this:
> [...]
> >     {
> >       camera: {
> >          up: [ 0, 1, 0 ],


> The  first thing bothering me is the combination of colons and brace characters
> in some places or maybe rather that it seems it can't be used the same  way
> elsewhere?
> Would we have to write camera: {
> It would seem a little cluttered to have to use a : for just specifying the
> opening of a block. do we have to ?

I would say that some of these things will have to be looked at from the
perspective of
1. the developer / parser
2. the end user

I would like it if we could dispense with some of the brackets altogether, and
just have a LF/CR/NL or new keyword signal the end of a statement.
On the other hand, I _would_ like to have explicit endings for code blocks, such
as
#endcamera
#endif
#endfor
#endwhile

as I think that overall it would be in the long run easier from both
perspectives to follow and debug code.

> > One possible change to the syntax could be the use of regular braces
> > around list items instead of square brackets to specifically denote
> > vectors, if only to make it more pleasant to read.

There has been some discussion about typing and automatic vector promotion.
Getting rid of brackets and braces altogether would make a little less typing to
do, but maybe it would be worth it to lose that benefit if we had to specify
things like vec2, vec3, vec4 - if only to keep it at the forefront of our minds
what sort of values we're dealing with in any given instance.


> > I'd also love to add ranges to the set of types, using a syntax akin to
> > this:
> >
> >     [ 1 .. 20 ] // range from 1 to 20, both inclusive
> >     ( 1 .. 20 ) // range from 1 to 20, both exclusive
> >     [ 1 .. 20 ) // range from 1 inclusive to 20 exclusive
> >     ( 1 .. 20 ] // range from 1 exclusive to 20 inclusive

Ranges would be very nice, but maybe I would like to get rid of the parentheses
in favor of a leading keyword, so that we could instead have all manner of
parentheses available for use in grouping terms in equations without making
parsing (more of) a complicated ordeal.

so, the keywords for the above examples might be ii, ee, ie, ei.

then we could write equations like   val = [sin (x+3) / pi] + (tau/6) -
abs{cos[(q/360)+(<n/0.5>+0.5)]};
It would also be nice to drop the requirement for #declare and just write
     x=3;


> > symbols as (entirely optional) syntactic sugar.

This would be very nice, especially for placing the text/symbol in the render.

The same would go for a mechanism for exposing the text of a parsed line of code
to the SDL.
What I mean by that is to have a mechanism similar to that in a spreadsheet
whereby if cell A3 has (x+1)/10 in it, then formula (A3) returns the string
"(x+1)/10"


> But with the prerequisites that they should have an explicit
> alternative for when we don't know or have any internet to check for the code to
> type. most people never enter a unicode special number their whole life. but
> maybe the parsing times could really be worth that learning?

We could do that with an include file, like we have with functions.inc -
symbol.inc could have #declare sym_nabla = symbol (U+2207); or however it would
get coded.

Post a reply to this message

Am 09.06.2021 um 20:13 schrieb Bald Eagle:

>>> I'd also love to add ranges to the set of types, using a syntax akin to
>>> this:
>>>
>>>      [ 1 .. 20 ] // range from 1 to 20, both inclusive
>>>      ( 1 .. 20 ) // range from 1 to 20, both exclusive
>>>      [ 1 .. 20 ) // range from 1 inclusive to 20 exclusive
>>>      ( 1 .. 20 ] // range from 1 exclusive to 20 inclusive
> 
> Ranges would be very nice, but maybe I would like to get rid of the parentheses
> in favor of a leading keyword, so that we could instead have all manner of
> parentheses available for use in grouping terms in equations without making
> parsing (more of) a complicated ordeal.
> 
> so, the keywords for the above examples might be ii, ee, ie, ei.
> 
> then we could write equations like   val = [sin (x+3) / pi] + (tau/6) -
> abs{cos[(q/360)+(<n/0.5>+0.5)]};
> It would also be nice to drop the requirement for #declare and just write
>       x=3;

I think we can have both.

The style of range notation is deliberately chosen to make it easy to 
determine what is the start of something and what the end; for instance, 
I personally prefer the following alternative notation because I find it 
more intuitive with respect to which end is "inclusive" and which is 
"exclusive", but _that_ would indeed seriously complicate parsing:

     [ 1 .. 20 ] // range from 1 to 20, both inclusive
     ] 1 .. 20 ] // range from 1 exclusive to 20 inclusive

As for the proposed syntax further above, distinguishing an arbitrary 
mathematical expression from a range should be easy: All the parser 
needs to do is look at the "..", which we can take as an operator that 
takes two numeric values and returns what we might call a "naked range". 
Parentheses and/or brackets around such a "naked range" would then 
convert that into what we might call a "qualified range". (Further 
wrapping a "qualified range" in more parentheses or brackets would have 
no additional effect.)

>> But with the prerequisites that they should have an explicit
>> alternative for when we don't know or have any internet to check for the code to
>> type. most people never enter a unicode special number their whole life. but
>> maybe the parsing times could really be worth that learning?
> 
> We could do that with an include file, like we have with functions.inc -
> symbol.inc could have #declare sym_nabla = symbol (U+2207); or however it would
> get coded.

Explicit ASCII alternatives, hard-baked into the language, would be a 
must, IMO. As I mentioned, Unicode symbols would be syntactic sugar. The 
ASCII constructs would be the real deal, while the Unicode symbols would 
be considered shortcuts.

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:

> Explicit ASCII alternatives, hard-baked into the language, would be a
> must, IMO. As I mentioned, Unicode symbols would be syntactic sugar. The
> ASCII constructs would be the real deal, while the Unicode symbols would
> be considered shortcuts.

Okay, and do you confirm that such kind of things would have significant impact
on parse time, like: linearly, if you divide the character amounts by two you
get half parsing code?

Post a reply to this message

Am 10.06.2021 um 13:18 schrieb Mr:
> clipka <ano### [at] anonymousorg> wrote:
> 
>> Explicit ASCII alternatives, hard-baked into the language, would be a
>> must, IMO. As I mentioned, Unicode symbols would be syntactic sugar. The
>> ASCII constructs would be the real deal, while the Unicode symbols would
>> be considered shortcuts.
> 
> Okay, and do you confirm that such kind of things would have significant impact
> on parse time, like: linearly, if you divide the character amounts by two you
> get half parsing code?

No, parser performance is not that simple.

A good parser (which POV-Ray's old one is not by any stretch, and even 
the overhauled one is only a step on the way there) will just _scan_ the 
whole file once (i.e. identify start and end of each character sequence 
that look like a token at first glance - e.g. sequences that look like 
numbers, sequences that look like keywords or identifies, sequences that 
look like operands, etc.), _tokenize_ it once (i.e. translate those 
character sequences into internal numeric IDs, aka tokens), and from 
there on just juggle those IDs.

The next steps would be to either...

- walk through those tokens and "execute" them, implementing loops by 
processing the corresponding tokens over and over again; in this case 
processing the loops again and again would be the bottleneck.

- digest that token sequence even further, "compiling" it into something 
that can be executed so efficiently that it might have a chance to 
become negligibe compared to the time spent scanning and tokenizing; but 
to achieve that, the effort to bring it into this efficient form will 
itself outweigh the effort of scanning and tokenizing.

In either case, the genuinely time-consuming portions of parsing will 
work on a representation in which the number of characters comprising 
the keywords or operands will have become entirely irrelevant.

Post a reply to this message

Bump! (unless discussion is going on elsewhere, if so, sorry for missing it) and
not knowing much about json yet, I have a remark / question to the experts.

I also like the idea to use end of lines for ending some statements. To me that
is one cause for the popularity of python, staying as close to pseudo code as
possible in the core functionality. (which doesn't prevent to use stuff like
brackets for list comprehensions or more recently type annotations, depending on
the scope of the snippet or project written. There's even some days when you
forget the colon after concentrating on formulation of an if clause, and you
think; couldn't we do without it?

However , I have to assume that some of the syntax previously mentionned is
organically intertwined within JSON core syntax... But I do not have the
experience to distinguish which at all.


So my question to the experts is, could end of line statement still be
compatible with the proposed json paradigm / standard ? maybe in the worse case,
breaking away from the standard but by chosing a subset of it... or whatever
shift...?

(I only asked because it would seem that we would be at leat two in favour of
such a feature, but I for one would gladly give the request up after being
explained a reason, such as performance gain etc...)

Post a reply to this message

Am 12.07.2021 um 15:01 schrieb Mr:

> So my question to the experts is, could end of line statement still be
> compatible with the proposed json paradigm / standard ? maybe in the worse case,
> breaking away from the standard but by chosing a subset of it... or whatever
> shift...?

One of the key features of the proposed language would be that any valid 
JSON file would automatically qualify as valid POV-Ray SDL file 
(provided the "document object model", aka "schema", matches, i.e. the 
hierarchy of how and what stuff is nested where).

Allowing for line endings to terminate statements might break that 
compatibility, because the JSON standard allows line endings anywhere it 
allows for whitespace.


I used JSON as a starting point for the proposed SDL format not only to 
start _somewhere_ but specifically because being 100% compatible with 
JSON would have the advantage that there are tools and libraries galore 
out there to generate that format.

JSON is one of just a handful of probably the top most popular formats 
for storage and exhange of structured data:

- XML
- JSON
- YAML

XML is painfully verbose, and therefore not an option. YAML is very 
concise, but assigns semantics not only to line endings but also to 
indentation, and that is something I'm anything but a fan of.

Which leaves us with JSON as the next most obvious choice - which 
happens to be similar to POV-Ray's current SDL both in terms of 
verbosity and overall look & feel (thanks to both ultimately being 
inspired by C).


Also, categorically making any line ending end a statement has the big 
drawback that any statement must be written on a single line. To work 
around this, the statement-ending semantics of line endings would have 
to be weakened depending on context, which in turn would add more 
complexity to the parser, and moreover to the language itself, 
potentially making it difficult for users to grasp how the line-ending 
rules work.

An alternative would be to make some of the commas optional, but at 
least as soon as we add certain of our own features we'd run into 
similar problems to avoid ambiguities.


> (I only asked because it would seem that we would be at leat two in favour of
> such a feature, but I for one would gladly give the request up after being
> explained a reason, such as performance gain etc...)

Compatibility with the base format, keeping the format free from 
ambigiuties, and keeping the format reasonably easy for users to grasp. 
That's pretty much all the reason there is to not assign special 
semantics to line endings.

In terms of performance, it probably wouldn't make much of a difference.

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> Am 12.07.2021 um 15:01 schrieb Mr:
>
> > So my question to the experts is, could end of line statement still be
> > compatible with the proposed json paradigm / standard ? maybe in the worse case,
> > breaking away from the standard but by chosing a subset of it... or whatever
> > shift...?
>
> One of the key features of the proposed language would be that any valid
> JSON file would automatically qualify as valid POV-Ray SDL file
> (provided the "document object model", aka "schema", matches, i.e. the
> hierarchy of how and what stuff is nested where).
>
> Allowing for line endings to terminate statements might break that
> compatibility, because the JSON standard allows line endings anywhere it
> allows for whitespace.
>
>
> I used JSON as a starting point for the proposed SDL format not only to
> start _somewhere_ but specifically because being 100% compatible with
> JSON would have the advantage that there are tools and libraries galore
> out there to generate that format.
>
> JSON is one of just a handful of probably the top most popular formats
> for storage and exhange of structured data:
>
> - XML
> - JSON
> - YAML
>
> XML is painfully verbose, and therefore not an option. YAML is very
> concise, but assigns semantics not only to line endings but also to
> indentation, and that is something I'm anything but a fan of.
[...]

Thanks a lot for this answer clarifying the stakes, Now I'll feel more at peace
when POV4 will keep its tolerant and meaningless line endings behaviour .

I would also hate it to be like xml compared to the other two options. So I
looked here https://levelup.gitconnected.com/json-vs-yaml-6aa0243aefc6


Coming from Python I know that my apriori not (yet) feeling the same about this
is probably thus biased, but could you please also develop more about why as a
user you would prefer an abundant punctuation over a meaningful indentation
system ?

Post a reply to this message

Am 13.07.2021 um 12:05 schrieb Mr:

> Coming from Python I know that my apriori not (yet) feeling the same about this
> is probably thus biased, but could you please also develop more about why as a
> user you would prefer an abundant punctuation over a meaningful indentation
> system ?

Indentation is trouble.

If you ever encounter a source file that mixes tab-indented and 
space-indented code, you may realize what I mean. In languages that 
don't assign semantics to indentation, such a mix results in eyesores at 
worst. In languages where whitespace matters, it can get far worse.

The core problem there of course is that there are different standards 
how many spaces a tab should be worth: Two? Four? Eight? Three? Six? 
Five? Been there, seen all of those.

And it's not like you can tell at a single glance what's going wrong 
there, with tabs and spaces both being entirely invisible and therefore 
looking identical.


And if you've ever edited makefiles (or proto-makefiles as used in 
Autoconf-based build systems), you really learn to hate languages that 
assign semantics to whitespace. If you set your editor to replace tabs 
with spaces, you _will_ break your makefiles.

(Granted, most indentation-aware languages don't assign different 
semantics to tab indentation vs. space indentation.)


So there's the tab issue, for starters.


The next issue of indentation-structured languages - or at least those 
that use a hard "end of line is end of statement" rule - is that they 
put a practical limit on how complex you can make a single statement. If 
you have a very complex mathematical formula, you need to break it up 
into smaller chunks, assign those to variables, and then assemble those 
into your final result. Or even more interim results first.

Now it can be argued that this forces you to give more structure to your 
code, thus making it easier to understand.

However, breaking up an expression into named sub-expressions is not the 
only way to structure complex formula. Specifically, even short formulae 
might benefit from being spread out over multiple lines. And maybe 
liberally peppered with indentation without having to worry about 
breaking semantics.

And what if you're just using a formula from someplace else, and have no 
idea whatsoever what the individual portions of the formula really mean? 
Or if you've optimized a mathematical expression for performance in such 
a way that its original "natural structure" is no longer visible anyway?

What you'll then do is use meaningless single-letter variable names to 
the sub-expression, and the resulting code will look far more complex 
and far more difficult to grasp than it would have, could you have 
written it as a single expression spread over multiple lines and 
indented freely.

And speaking of variable names, restricting the practical length of a 
statement may actually lead to poorer rather than better code, in that 
people may develop a habit of keeping variable names short and thus 
potentially more cryptic.

(Some "end-of-line-aware" programming languages allow spreading 
statements across multiple line by "invalidating" a line ending, but the 
corresponding syntax tends to be somewhat clunky. If you've ever done 
much C/C++ preprocessor programming, you'll know what I mean. What's 
more, such syntax often breaks in the presence of stray blanks at the 
end of a line, which are a pain to find.)


Also, sometimes it may actually improve readability to put multiple 
short statements on a single line (especially when there's a group of 
statements that repeats in structure). Some end-of-line-aware languages 
allow this (e.g. shell scripts), but again generally at the cost of 
added symbols and therefore clunkiness.

Of course it can be argued that whitespace-agnostic languages need the 
same extra symbols between statements, not only in a single line but 
also between lines. But in my mind separator characters become less 
clunky if they're used consistently throughout, as opposed to just on 
special occasions.


And then there's the matter of copy-and-pasting:

When you copy-and-paste code from arbitrary sources (a web page here, a 
PDF there, a terminal window over there, and what else you might happen 
to come across), you may end up with broken indentation.

If the code is reasonably complex, have fun faithfully restoring the 
original semantics.

If the semantics is all in the non-whitespace, smart editors may 
actually be able to restore the original indentation, at least to the 
point that its structure is agion easy enough to grasp for a human. If 
semantics is in the indentation itself, how could an editor ever restore 
that indentation?


Also, look at written human language: There's a reason we're using 
special characters to indicate the boundaries between sentences and 
subclauses, rather than using a "one line = one sentence" default rule 
and using special syntax to spread sentences across multiple lines.


Now I can't rule out that Python may for some reason manage to get 
around each and every one of the above issues - I've had virtually no 
exposure to that language yet. But based on my experience with other 
end-of-line- and indentation-aware languages, my guess is that at least 
some of those issues do apply.

But even if Python should be entirely immune, that doesn't mean it would 
be easy to design a new POV-Ray SDL that is also indentation-aware and 
has the same level of immunity as Python.

And yet, feel free to throw together a corresponding proposal, and 
present it here. Even if it should turn out to be seriously flawed, 
there may still be interesting and inspiring morsels in there.


Heck, I've never claimed that my own proposal was really any good. I 
just threw it out there to have _some_ starting point. For discussion at 
worst, or for further development at best.

As a matter of fact, I don't even _particularly_ like it. It's not very 
elegant, because it is not custom-tailored to our needs. It is derived 
from an existing base. It does have interesting properties though, one 
of which arises _specifically_ out of being derived from an existing 
base. And for now I think it has the potential to be useful, and maybe 
even reasonably good.

No worse than the current SDL, and better structured - that's all I'm 
asking of a new SDL. Ideally with consistent syntax between scene 
descriptions and functions, and maybe with a rigorous enough structure 
to allow for a compiling rather than interpreting parser.

Post a reply to this message

hi,

clipka <ano### [at] anonymousorg> wrote:
> ...
> No worse than the current SDL, and better structured - that's all I'm
> asking of a new SDL. Ideally with consistent syntax between scene
> descriptions and functions, and maybe with a rigorous enough structure
> to allow for a compiling rather than interpreting parser.

have you looked at Tcl/Tk yet, for inspiration?  has many (imo) natty features,
nice, clean syntax, and byte-compiles.


regards, jr.

Post a reply to this message

in news:60eda74b@news.povray.org clipka wrote:

> Heck, I've never claimed that my own proposal was really any good. I 
> just threw it out there to have _some_ starting point. For discussion
> at worst, or for further development at best.
> 

Lets go wild then. I don't like it. I like how the current SDL is 
sufficient. Fit for use. Although there's a lot I would leave out. 
#declare, #local be gone! Proper, simple, understandable namespaces and 
scoping with maybe a (exportable) global. All the #'s be gone. What's 
left then is quite clean. Oh ';' out, out out. Squiggels is enough and 
(),[],<>. Vectors are great.

Then there's a few little gripes "for (I, 0, dimension_size(ArrayX, 1)-
1)" for example. I know the dimension_... comes from the multidimensional 
arrays, but are they still needed when we have arrays of arrays? "for (I, 
0, size(ArrayX)-1)" or "for (I, 0, size(ArrayX[5])-1)".

So, no brand new syntax for me, but a cleaned up version as close as 
possible in functionality as the current one. No OOP. Would iterators, 
generators make sense? I doubt it. No async. Minimal to no functional 
stuff. Also something that relative easily can be converted from old to 
new (probably impossible given how the SDL can be 'abused').

Under the hood? I would like to have more access to all kind of variables 
POV-Ray calculates so we can create 'things' (textures) depending on 
those results.

Ok, now for the crazy one: Wasm. https://webassembly.org/
I like how the compiled functions in math.inc work through an include 
file. I like the ideas of adding more of those. 
I even more like the idea if we could write dynamic linked libraries, 
call them from POV-Ray using a compagnion include file. (something 
simmilar is done in SuperCollider c++ for the audio server and a *.sc 
file for the scripting side). 
Now, what if these wasm's can be used as 'dll's'. It would make them 
platform independent plug-in's. Reduction of maintenance. There are a lot 
of languages that can be compiled this way, proper toolchains et al.

For what? Not directly for interacting with the ray's, if possible at 
all. But for easy access to for example nurbs (Rhino OpenNurbs) of faster 
mesh generation or boning and IK and surface subdivison, 
http://math.lbl.gov/voro++. Simply put, more extensive and faster include 
files (for things you don't want in the core).
Or, the other way around. Could it be made possible to convert a parsed 
file into 3D printer data more easily, with checks for holes we don't 
need when tracing? OGL visualisations?

Ingo
-- 
https://ingoogni.nl

Post a reply to this message

Am 14.07.2021 um 11:35 schrieb jr:
> hi,
> 
> clipka <ano### [at] anonymousorg> wrote:
>> ...
>> No worse than the current SDL, and better structured - that's all I'm
>> asking of a new SDL. Ideally with consistent syntax between scene
>> descriptions and functions, and maybe with a rigorous enough structure
>> to allow for a compiling rather than interpreting parser.
> 
> have you looked at Tcl/Tk yet, for inspiration?  has many (imo) natty features,
> nice, clean syntax, and byte-compiles.

Unless those features have a direct impact on the core principles of the 
language's syntax, I don't care much for them at this point. Also, I 
think it would be more productive to name those features that you like 
about it, rather than just (to put it into more blunt of a picture than 
adequate here) point in the general direction of the language and say, 
"I like some of the stuff that's in there, see if you can spot it".

As for nice, clean syntax - I would disagree.

At a very first glance, Tcl/Tk sounds like a promising basis: Its main 
purpose (nowadays) is also to describe hierarchical structures (albeit 
not geometric objects and surface properties, but rather GUI elements), 
with programmatic elements mixed in for good measure. Unfortunately, the 
Tk part (which is the description of GUI elements) was bolted on later, 
and in my opinion it shows. At its very core, Tcl is a command 
shell-style scripting language.

One of the results is that different Tcl statements may have their own 
individual quirky "micro-syntax". Just like all of POV-Ray's individual 
geometric primitive, pattern and what-have-you statements all have their 
own little "micro-syntax" quirks. Any commonalities between them are 
just superficial conventions. Which bothers me to no end.


In my opinion, the best approach to judge a language for suitability as 
the basis for a new SDL is to ask (and answer) the following three 
questions:

- "What is the 'grammar' of this language, i.e. the core principles that 
govern its syntax?"

And based on that:

- "How would a simple scene be described in this language?"
(A chrome sphere on a colorful checkered plane, with camera and all, and 
maybe a text object to showcase strings, should do as a first example.)

- "How would programmatic elements mesh with this syntax?"
(A simple example with a loop and a few conditionals should be a 
reasonable starting point.)


When I do that for Tcl/Tk, I find that the results look - in my mind - 
ugly AF.

Post a reply to this message

Am 14.07.2021 um 20:15 schrieb ingo:

> Lets go wild then. I don't like it. I like how the current SDL is
> sufficient. Fit for use. Although there's a lot I would leave out.
> #declare, #local be gone! Proper, simple, understandable namespaces and
> scoping with maybe a (exportable) global.

So there _is_ something you would add: Namespaces.

Also, for something that is "fit for use", there's quite a lot you would 
seem to change ;)

> All the #'s be gone. What's
> left then is quite clean.

But is it though?

Trim the `#` from the `#version` statement, and it clashes with the 
pseudo-variable `version`, for example. Trim the `#` from the `#local` 
statement, and it clashes with the `local` pseudo-dictionary.

You're asking for namespaces - you can look at `#` as the namespace for 
control statements.


(There's also one particular property the use of `#` has, that is 
invaluable at least for the current parser:

It helps a lot in speeding up the parsing of some control structures. 
For example, after encountering `#if()` with an expression that is 
false, the parser can simply skip from `#` to `#` in search for the 
matching `#else` or `#end`, because anything else is guaranteed to not 
natter. Likewise, after encountering `#macro(...)`, the parser can 
simply skip from `#` to `#` in search for the matching `#end`, because 
it will only parse the macro body once it is invoked.

Of course that may be less of an issue once we change the parsing 
strategy to scan and tokenize the file only once, and then work on the 
"pre-digested" token sequence.)


> Oh ';' out, out out. Squiggels is enough and
> (),[],<>. Vectors are great.
> 
> Then there's a few little gripes "for (I, 0, dimension_size(ArrayX, 1)-
> 1)" for example. I know the dimension_... comes from the multidimensional
> arrays, but are they still needed when we have arrays of arrays? "for (I,
> 0, size(ArrayX)-1)" or "for (I, 0, size(ArrayX[5])-1)".

Genuine multidimensional arrays are easier to set up:

     #declare Foo = array[10][10];

as opposed to:

     #declare Foo = array[10];
     #for (I, 0, 9)
         #declare Foo[I] = array[10];
     #end

Also, they guarantee that the size of the second dimension is constant, 
whereas arrays of arrays might have different sizes in the second dimension.

They are also more memory-efficient, especially if the last dimension is 
comparatively small.

And I guess they are also more efficient in terms of performance, 
although that might not show much in comparison to the general slowness 
of the current parser.


> So, no brand new syntax for me, but a cleaned up version as close as
> possible in functionality as the current one.

> No OOP.

I hope you'll be fine with structured data aggregates at the very least 
(think POV-Ray v3.8's dictionaries).

And since dictionary entries can be functions (or at least should be 
able to; not 100% if that currently works), and I'm determined to make 
functions more powerful (wouldn't you want to be able to write tiny 
little programs in functions, if only to more easily work with interim 
results?), OOP will probably enter the scene as a side effect.

If it weren't for the special syntax of macros (`#macro Foo() ... #end` 
instead of `#declare Foo = macro() ... #end`)

Also, my goal is to formalize the object, pattern and whatnot syntax 
more, which will feature OOP at least under the hood. And wouldn't it be 
nice to implement custom objects that look and feel just like the 
inbuilt ones, syntax-wise?

> Would iterators,
> generators make sense? I doubt it.

I'm not quite so sure. Unless we provide a goot set of container-style 
inbuilt data structures (which _are_ useful after all), people will 
start to use structured data aggregates to roll their own, and end up 
with the desire to use them in a nice elegant `for myElement in myArray` 
(or alike) loop. That'll be difficult to implement generically without 
the concept of an iterator.

The good news is that iterators are just a matter of proper 
implementation, once you provide OOP basics.

> No async.

Sure? It would allow users to leverage multiple CPUs even during parsing.

> Minimal to no functional
> stuff.

If you mean syntax to support functional programming, I agree. I can't 
wrap my head around that paradigm anyway, so I doubt I could design or 
even implement a language to support it properly.

If you mean powerful syntax for functions, I disagree. I do want at 
least the ability to easily compute interim results in functions.

> Also something that relative easily can be converted from old to
> new (probably impossible given how the SDL can be 'abused').

You're probably right there: It would be a pain to write a converter. 
Because it is a pain to write a parser for the current SDL in the first 
place.

> Under the hood? I would like to have more access to all kind of variables
> POV-Ray calculates so we can create 'things' (textures) depending on
> those results.

Ooooh... so maybe some OOP features after all?

Also: Can you be a _little_ bit more specific?

There's a lot of stuff POV-Ray calculates under the hood, but only a 
fraction thereof can reasonably accessed from SDL.

> Ok, now for the crazy one: Wasm. https://webassembly.org/
> I like how the compiled functions in math.inc work through an include
> file. I like the ideas of adding more of those.
> I even more like the idea if we could write dynamic linked libraries,
> call them from POV-Ray using a compagnion include file. (something
> simmilar is done in SuperCollider c++ for the audio server and a *.sc
> file for the scripting side).
> Now, what if these wasm's can be used as 'dll's'. It would make them
> platform independent plug-in's. Reduction of maintenance. There are a lot
> of languages that can be compiled this way, proper toolchains et al.

I don't think that will happen, at least not anytime soon. Allowing for 
arbitrary binaries to be plugged in can open up quite a can of worms, 
security issues being only one of them.

Also, I'm sure such plug-ins _will_ create intersections with the world 
of OOP.

> Or, the other way around. Could it be made possible to convert a parsed
> file into 3D printer data more easily, with checks for holes we don't
> need when tracing? OGL visualisations?

Possibly, eventually. Long road there. But that's less of a parser 
matter, and more of a render pipeline thing. We need to pull that apart 
in order to more easily provide SDL-level access to its elements.

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> Also, I
> think it would be more productive to name those features that you like
> about it, rather than just (to put it into more blunt of a picture than
> adequate here) point in the general direction of the language and say,
> "I like some of the stuff that's in there, see if you can spot it".


> One of the results is that different Tcl statements may have their own
> individual quirky "micro-syntax". Just like all of POV-Ray's individual
> geometric primitive, pattern and what-have-you statements all have their
> own little "micro-syntax" quirks. Any commonalities between them are
> just superficial conventions. Which bothers me to no end.
>
>
> In my opinion, the best approach to judge a language for suitability as
> the basis for a new SDL is to ask (and answer) the following three
> questions:
>
> - "What is the 'grammar' of this language, i.e. the core principles that
> govern its syntax?"

As I (and probably most of us) have never really dissected, analyzed, and
evaluated a computer language like that, let alone several - to be able to
compare and contrast - this is likely going to be tough issue to address.
Maybe provide an example or a link to exactly the kind of thing you're talking
about here.

> And based on that:
>
> - "How would a simple scene be described in this language?"
> (A chrome sphere on a colorful checkered plane, with camera and all, and
> maybe a text object to showcase strings, should do as a first example.)
>
> - "How would programmatic elements mesh with this syntax?"
> (A simple example with a loop and a few conditionals should be a
> reasonable starting point.)

Now, I can only chime in here with the things I've seen.
OpenGL and Shadertoy seem to have all/most of the things that we want, and
OpenGL is well established, and presumably geared specifically towards graphics.
;)
It seems like a fairly easy and relatively intuitive language for anyone who's
done "regular" basic programming, and played with some of the patterns and
isosurface/parametric stuff.   It seems to be compiled, fast, and have really
user-friendly function and vector features, which would be essential.

I've seen people program things in c++ and python, and I kinda hate it.
I've seen some stuff in Processing and P5JS, but only a very limited amount.
I've heard mention of Lua and Haskell...  :D

Some of things I like about the OpenGL and P5JS work, is that there's a looping,
continually updating output - somewhere - and I wish we had the capability to
have  "a running program that outputs graphics" in place of / in addition to the
2-step edit-then-render a single static image paradigm that we have.

I guess since the languages exist and there are plenty of examples, the syntax
issue can be left to folks better informed than I, but the last two questions
can be seen in action, where objects, light sources, shadows, reflection models,
geometry, and program control directives are all used to create a scene.

In my head, I'm envisioning that an interim "meta-POV-Ray" could be implemented,
where everything is written in user-accessible include files, and a "free for
all with rules" ensues, where people can "access the source code" (the include
files) and edit it to produce new objects, algorithms, reflection models,
geometries, and even object and command syntaxes.  People can then have their
own "anything goes" forks, but team contributions for serious consideration need
to adhere to certain established conventions.   Then a sort of natural evolution
could be allowed to play out, and the various strengths and weaknesses of each
approach could be tested by the community as a whole.

We could have whole algorithms to generate shapes and textures rather than
functions, persistent variables across renders, and a host of other things I
likely can't think of off the top of my head.

The interesting parts that I see about this approach, are that (lets take
Shadertoy as a specific example) the "core" of the application isn't
intermingled with the _things_ that the core handles.  It's a programming
language who's end product is primarily a colored pixel on the screen, but all
of the basic primitives lie outside of the core and are just macros of sorts.

As such, it's wide open to the user to do as they please with.  That takes a lot
of development pressure off the devs and lets users so inclined to code up
whatever cameras, lights, shapes, and patterns that they can think of.  It also
addresses one of the issues that constantly pops up on the forums about "Why
can't POV-Ray...", because then it's easy to point them to the include file
they're invoking for - let's say a shape.  It may be written with very
meaningful variable names or extensively commented, and the actual code that
makes the shape they are interested in is right there for them to see.  It's a
wonderful learning experience for people interested in computer graphics.

POV-Ray's pigment pattern equations are beautiful in their simplicity, and some
of the more complex ones are a wonder to behold.  I'm often sad that the logic
of the pigment patterns isn't up front in the documentation, because it gives a
new user a WAY to think about exactly how the magic on the screen happens.

Most of the people "out there" are POV-Ray _users_, and as you have pointed out,
may only use it as the last link in a software workflow chain.  Most of use
_here_ are "only" users, and not developers who have a good-enough grasp of the
source code and how it all fits together to get to the part that colors the
current pixel.  Something that is still usable but draws the more-than-casual
user into deeper quasi-developer territory would, in my opinion, help address a
lot of the things that we are struggling with.
Basic understanding of "how it all works"
Educating people with programming and mathematical aptitude to make parts of
POV-Ray their own and put on the developer hat.
Greater freedom to edit the source, and a nimbler evolution of future versions.
Less reliance on only 2 or 3 people who know how anything works to a great
enough degree to actually make or change something without breaking the rest of
it.
The addition of new features to POV-Ray, since greater experience with writing
"SDL" de facto translates to writing POV-Ray "source code".
In the case of Shadertoy, it's a browser-driven software, which makes it
inherently cross-platform.


Obviously, there are going to be problems with any of the points I made, and
some are likely painfully naive to the experienced software developer, and
especially to you and others who have a top-down view of the whole inner
workings of the source code and the interplay of all the dependencies - but I'm
just sharing my ideas in this rap session.  Sometimes some of them actually turn
out not to suck.  ;)

Post a reply to this message

Am 15.07.2021 um 02:12 schrieb Bald Eagle:


> OpenGL and Shadertoy seem to have all/most of the things that we want, and
> OpenGL is well established, and presumably geared specifically towards graphics.
> ;)

I have to ask what exactly you are referring to when you say "OpenGL". 
At its core, OpenGL is a library (or, more specifically, a library API): 
A collection of functions that can be called from various entirely 
different programming languages. The most "native" of those would be C/C++.

Are you perhaps referring to the shader language of OpenGL (GLSL)?

If you love GLSL, then I'm surprised you "kinda hate" C++.

Also, I'm not quite sure how a POV-Ray scene could map to GLSL. It seems 
to me like it would have to be described in as clunky a way as it would 
in C++.


In essence, most program languages have an imperative approach at the 
lowest level; to describe data, you have to instruct the computer to 
assemble it step by step, like so (pseudocode):

     Create a sphere S.
     Set the radius of S to 5.
     Set the center of S to <0,1,0>.

     Create a texture T.

     Create a pigment P.
     Set the pigment's pattern to 'checker'.
     ...
     Attach P to T.

     ...
     Attach T to S.
     Attach S to the scene.

Rather than using a descriptive approach, like so:

     The scene has the following objects:
     - A sphere, with the following properties:
       - a radius of 5.
       - a center of <0,1,0>.
       - a texture with the following properties:
         - a pigment with the following properties:
           - pattern 'checker'

While the difference may not seem obvious at first glance, it tends to 
make a huge difference in terms of how much repetitive stuff the syntax 
requires. For example, in C++ it might be written like so:

     Object S = new Sphere();
     S.radius = 5;
     S.center = Vector3(0,1,0);
     Texture T = new Texture();
     Pigment P = new Pigment();
     P.pattern.set('checker');
     ...
     T.addPigment(P);
     ...
     S.addTexture(T);
     Scene.addObject(S);

whereas in a descriptive language it might be as simple as:

     sphere {
       radius 5
       center <0,1,0>
       texture {
         pigment {
           pattern 'checker'
           ...
         }
         ...
       }
     }

Post a reply to this message

hi,

clipka <ano### [at] anonymousorg> wrote:
> Am 14.07.2021 um 11:35 schrieb jr:
> > have you looked at Tcl/Tk yet, for inspiration?  has many (imo) natty features,
> > nice, clean syntax, and byte-compiles.
> ...
> At a very first glance, Tcl/Tk sounds like a promising basis: Its main
> purpose (nowadays) is also to describe hierarchical structures (albeit
> not geometric objects and surface properties, but rather GUI elements),
> with programmatic elements mixed in for good measure. Unfortunately, the
> Tk part (which is the description of GUI elements) was bolted on later,
> and in my opinion it shows. At its very core, Tcl is a command
> shell-style scripting language.

yes and no.  I'd argue that the interpreter is the core.  and that is designed
to be extended (with "package"s and other mechanisms), and designed to be
embedded, that is provide scripting support to an application.


> One of the results is that different Tcl statements may have their own
> individual quirky "micro-syntax". Just like all of POV-Ray's individual
> geometric primitive, pattern and what-have-you statements all have their
> own little "micro-syntax" quirks. Any commonalities between them are
> just superficial conventions. Which bothers me to no end.

would not, in the end, a database "backbone" allow to, bit by bit, reduce the
"chaos"? (for instance, the stuff provided in the many /include/ files, if
pre-processed and stored in a db, could then allow the parser to simply "paste"
the code actually used, on demand so to speak)


> In my opinion, the best approach to judge a language for suitability as
> the basis for a new SDL is to ask (and answer) the following three
> questions:
>
> - "What is the 'grammar' of this language, i.e. the core principles that
> govern its syntax?"
>
> And based on that:
>
> - "How would a simple scene be described in this language?"
> (A chrome sphere on a colorful checkered plane, with camera and all, and
> maybe a text object to showcase strings, should do as a first example.)
>
> - "How would programmatic elements mesh with this syntax?"
> (A simple example with a loop and a few conditionals should be a
> reasonable starting point.)
>
>
> When I do that for Tcl/Tk, I find that the results look - in my mind -
> ugly AF.

agree that "just translating" does not work too well.  and the fact that Tcl
does not provide "traditional" arrays is another drawback.  still (biased and
all) I think that owing to its design, a SDL/Tcl could make a "nice hybrid".


regards, jr.

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:

> to allow for a compiling rather than interpreting parser.

I'm so glad I asked the question ! Now not only am I fully convinced by all of
your points ! But I
actually am pretty enthusiastic ! Thanks to the last line,  about your proposal:
compiled POV scenes, wow ! that sure is a great benefit. And just to make sure I
didn't get you wrong, this means some speed increase too, doesn't it?

Post a reply to this message

Am 16.07.2021 um 15:35 schrieb Mr:
> clipka <ano### [at] anonymousorg> wrote:
> 
>> to allow for a compiling rather than interpreting parser.
> 
> I'm so glad I asked the question ! Now not only am I fully convinced by all of
> your points ! But I
> actually am pretty enthusiastic ! Thanks to the last line,  about your proposal:
> compiled POV scenes, wow ! that sure is a great benefit. And just to make sure I
> didn't get you wrong, this means some speed increase too, doesn't it?

When re-running one and the same scene a lot, say, in an animation? Yes, 
most certainly.

When looping a lot in a scene, even if you render it only once? Probably.

When using a lot of include files that you tend to use over and over 
again? Or rendering a scene again that is structured into many 
individual files, and you've changed only a single one of them? We could 
probably get some performance win for that scenario, too.


When rendering a monolithic still scene, with very few programmatic 
elements, just that one single time? No, most likely not.


So we should have a way for the user to tell the parser which files to 
compile, and which just to interpret.

Post a reply to this message