Some of these are things I have discussed years ago, but I thought it
could be useful to revive the discussion a bit.

One of the original plans for POV-Ray 4 was to make a complete rehaul
of the SDL from scratch, making it a lot more versatilve, expressive,
powerful and faster (via byte-compilation). This would have been a
completely new input language that's different from the current one
(but with, perhaps, a backwards compatibility mode to support existing
scene files.)

I have no idea what the current plans are, but I have the impression
that the idea of this kind of complete overhaul has been pretty much
scratched, in favor of simply enhancing the current SDL. (If I'm wrong,
I'd love to hear about the current development ideas on this.)

If we are simply going to keep the current SDL and enhance it, rather
than create a completely new input language, perhaps a kind of in-between
solution could be possible.

I really think that the enhancements made to the SDL should use the same
design principle as what happened with graphics accelerator cards.

In the late 90's and very early 2000's, when graphics accelerator cards
were becoming more and more popular, and were becoming more and more
advanced, the manufacturers were doing the same thing as the POV-Ray
SDL has been doing: Piling up individual hard-coded features one on
top of another. Basically everything that graphics cards supported at
the time were such individual hard-coded, inflexible features (such
as lightning, bump maps, vertex transforms, etc.)

Quite soon, however, graphics card manufacturers realized that this is
a completely infeasible design. It needlessly limits what games can do,
it makes the graphical options limited and very rigid, and it stifles
the game developers' options. New graphical effects can't be added to
the game if the graphics card doesn't directly support it.

Thus they went into a rather different direction: Rather than add more
and more fixed features, they instead concentrated their efforts in
implementing an expressive fully-fledged efficient machine code, which
could be run on a vertex-by-vertex and on a pixel-by-pixel basis
(namely, vertex and pixel "shaders"). This machine code could be given
data and parameters both by the running program and the graphics card
itself, and it could do whatever calculations with them (and pass that
data from the vertex shader to the pixel shader as needed) which would
affect the coloration of the final pixels.

Thus, for example, rather than add yet another type of rigid, fixed form
of, say, bump mapping, this machine code system would allow the game
developers to implement their own freely customized version. (This
freely-implementable "bump mapping" could even be algorithmically
generated and even animated, if so desired. There is basically no limit.)

This design has been highly successful. Rather than spend time and effort
on fixed features, they have spent that time and effort in making the
machine code system more efficient and even more flexible. (This is such
a crucial feature of current graphics cards that their prowess is pretty
much solely measured in how many shader operations per second they can do.)

Now, I'm not saying that POV-Ray should implement a "shader" language
that can do absolutely anything (although if well designed and implemented,
that would be great). However, I think that future expansion of the
current SDL should, in my opinion, at least go more into that direction.

A concrete example: I have suggested in the past a new feature for POV-Ray:
A kind of "poor man's blurred reflection" system, where the reflection on
a surface is the more faded the farther away the reflected surface is.
In other words, the strength of the reflected ray's color is inversely
proportional to the length of said ray. (When used properly, this can
give a kind of "blurred reflection" effect, with the advantage over actual
blurred reflection that this would be significantly faster. Imagine, for
example, this effect used on some porcelain object with a weak reflection
to start with.)

However, rather than add this particular effect as a fixed, hard-coded
feature, it would be much better if the user would be given the option
of specifying some kind of function for eg. how the contribution of
reflected rays are calculated on the color of a surface. This way it
would not only be possible to implement this particular feature, but
a myriad of other features. (This user-defined function should, obviously,
get as parameters, or be able to get with special functions, all kinds of
info about the reflected ray, the reflecting object, the reflected object,
and other data.)

This doesn't necessarily have to have an impact on rendering speed when
*not* using the user-defined-function-for-reflections feature. The
implementation could simply check if such a function has been specified,
and if not, then use the old method. This should have a negligible impact
on rendering speed.

These user-defined functions should of course be used in other places
as well, not just in reflected ray calculations.

As said, I think that enhancements made to the current SDL should at
least try to go more into this direction, rather than just keep adding
more and more new fixed hard-coded features.

-- 
                                                          - Warp

Post a reply to this message

Warp <war### [at] tagpovrayorg> wrote:
> Some of these are things I have discussed years ago, but I thought it
> could be useful to revive the discussion a bit.
>
> One of the original plans for POV-Ray 4 was to make a complete rehaul
> of the SDL from scratch, making it a lot more versatilve, expressive,
> powerful and faster (via byte-compilation). This would have been a
> completely new input language that's different from the current one
> (but with, perhaps, a backwards compatibility mode to support existing
> scene files.)
>
> I have no idea what the current plans are, but I have the impression
> that the idea of this kind of complete overhaul has been pretty much
> scratched, in favor of simply enhancing the current SDL. (If I'm wrong,
> I'd love to hear about the current development ideas on this.)
>

If we go this way, I would like to see some object-oriented capabilities (e.g.
the ability to create an object and instantiate it multiple times with different
attributes rather than wrapping it in another object { } block)

I would also like to see recursive function definitions, but not knowing how
complex this is, I consider it secondary.

> If we are simply going to keep the current SDL and enhance it, rather
> than create a completely new input language, perhaps a kind of in-between
> solution could be possible.
>

The problem would be that any major enhancement to the existing system is almost
certain to be a rewrite anyway, so why not use a new language (except that it
would completely break old scenes and necessitate rewriting current projects.)

> Snip (History lesson)

the problem I see with this is that not all of us are good at this sort of
thing.  If this is the direction povray were to go, then I think it would be
better to have a plugin system so that, rather than compiling dozens of
unofficial versions of POVRay, we could have One Ring to Rule Them All, and
extra functionality -- even overriding functionality -- could be handled via the
plugin system.

Regards,
A.D.B.

Post a reply to this message

Am 30.07.2015 um 12:15 schrieb Warp:

> One of the original plans for POV-Ray 4 was to make a complete rehaul
> of the SDL from scratch, making it a lot more versatilve, expressive,
> powerful and faster (via byte-compilation). This would have been a
> completely new input language that's different from the current one
> (but with, perhaps, a backwards compatibility mode to support existing
> scene files.)
>
> I have no idea what the current plans are, but I have the impression
> that the idea of this kind of complete overhaul has been pretty much
> scratched, in favor of simply enhancing the current SDL. (If I'm wrong,
> I'd love to hear about the current development ideas on this.)

The idea is not scratched at all. It's a big endeavour though, compared 
to the time available, and some other fancy stuff can't wait to get 
included (or, rather, I can't wait to include it ;-)), so it needs to be 
woven into the current SDL.


> Now, I'm not saying that POV-Ray should implement a "shader" language
> that can do absolutely anything (although if well designed and implemented,
> that would be great). However, I think that future expansion of the
> current SDL should, in my opinion, at least go more into that direction.

I'm well aware of your wish for this road to be taken, and while it's 
currently invisible in the SDL, under the hood things are already moving 
in that direction (your past posts have been quite inspirational in this 
respect). More and more portions of the code get clearly delineated from 
each other, modularized, and shoved into well-defined polymorphic 
classes - which will make it a whole lot easier to plug in user-defined 
code.

One of the next steps on the agenda, for instance, will be to smash the 
basic shading code into handy congruent pieces; under the hood, a single 
texture layer will then not have one phong, one blinn-phong and one 
diffuse component, but rather just have an arbitrary bunch of basic 
shading effects, each of which can be phong, blinn-phong, diffuse, or 
any of a number of more sophisticated models yet to be implemented - one 
of which could be a wrapper for user-supplied code.

Initially, the parser will limit the power of this change by retaining 
the current syntax, and just mapping it to the new internal model behind 
the scenes, but I can imagine extending the current SDL syntax to breach 
the current limit of number and type of effects soon after.


However, the user-supplied code portion of the scheme - and actually any 
user-supplied code - /will/ have to wait for the parser rewrite: 
User-supplied code needs /some/ sufficiently expressive syntax, a parser 
for that, and a sufficiently powerful VM to run on - and I guess we 
agree that it makes perfect sense to make that identical to the new 
syntax, parser and VM for the general scene description.

Post a reply to this message

Am 31.07.2015 um 06:37 schrieb Anthony D. Baye:

> If we go this way, I would like to see some object-oriented capabilities (e.g.
> the ability to create an object and instantiate it multiple times with different
> attributes rather than wrapping it in another object { } block)

That's what I want (and intend) to see, too.

> I would also like to see recursive function definitions, but not knowing how
> complex this is, I consider it secondary.

Once we throw out our current function VM and replace it with LLVM, that 
should be possible.

BTW, one of the most recent changes (1b5aab6) involved abstracting the 
current function VM out of the core code. From the render engine's point 
of view, LLVM could be introduced any moment now - "only" the parser 
would have to be adapted.

Post a reply to this message

Anthony D. Baye <Sha### [at] spamnomorehotmailcom> wrote:
> the problem I see with this is that not all of us are good at this sort of
> thing.  If this is the direction povray were to go, then I think it would be
> better to have a plugin system so that, rather than compiling dozens of
> unofficial versions of POVRay, we could have One Ring to Rule Them All, and
> extra functionality -- even overriding functionality -- could be handled via the
> plugin system.

My suggestion had nothing to do with compiling POV-Ray, or having a plugin
system. It was about enhancing the current scene decription language with
features that are closer to shaders in design.

Current user-defined functions are already a step in that direction.
My suggestion was to expand that idea and make it usable in the actual
rendering (in different parts of it.)

And you wouldn't need to know how to use it, or the full language.
Those who would know, could make easy-to-use library functions that
you can use (in the same way as the current include system works.)

This has many advantages. For instance, if you needed a functionality
that's similar to what somebody else has done, you can just take that
"shader" of theirs and modify it a bit to suit your needs. No need to
recompile POV-Ray or use a complex "plug-in" system. Just write some
SDL and that's it.

-- 
                                                          - Warp

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> I'm well aware of your wish for this road to be taken, and while it's 
> currently invisible in the SDL, under the hood things are already moving 
> in that direction (your past posts have been quite inspirational in this 
> respect). More and more portions of the code get clearly delineated from 
> each other, modularized, and shoved into well-defined polymorphic 
> classes - which will make it a whole lot easier to plug in user-defined 
> code.

I'm glad to hear that.

> However, the user-supplied code portion of the scheme - and actually any 
> user-supplied code - /will/ have to wait for the parser rewrite: 
> User-supplied code needs /some/ sufficiently expressive syntax, a parser 
> for that, and a sufficiently powerful VM to run on - and I guess we 
> agree that it makes perfect sense to make that identical to the new 
> syntax, parser and VM for the general scene description.

Do you mean that the "shader" code ought to have identical syntax to
everything else in the input language (rather than them being essentially
two separate and independent languages, one inside the other, like the
current SDL and the user-defined functions are now)?

-- 
                                                          - Warp

Post a reply to this message

Am 02.08.2015 um 00:43 schrieb Warp:

> I'm glad to hear that.
>
>> However, the user-supplied code portion of the scheme - and actually any
>> user-supplied code - /will/ have to wait for the parser rewrite:
>> User-supplied code needs /some/ sufficiently expressive syntax, a parser
>> for that, and a sufficiently powerful VM to run on - and I guess we
>> agree that it makes perfect sense to make that identical to the new
>> syntax, parser and VM for the general scene description.
>
> Do you mean that the "shader" code ought to have identical syntax to
> everything else in the input language (rather than them being essentially
> two separate and independent languages, one inside the other, like the
> current SDL and the user-defined functions are now)?

Yup.

Note that my goal is for the language /per se/ to have a very simple 
syntax, and make it pretty much oblivious to rendering, except for 
providing a few special native data types like 3D vectors and colours. 
It'll be up to a bunch of predefined classes to fill it with raytracing 
life in its scene description role. Such a language should be generic 
enough to also be suited for the shader description role, so it would be 
rather pointless and user-unfriendly to devise yet another language for 
that purpose.

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> Note that my goal is for the language /per se/ to have a very simple 
> syntax, and make it pretty much oblivious to rendering, except for 
> providing a few special native data types like 3D vectors and colours. 
> It'll be up to a bunch of predefined classes to fill it with raytracing 
> life in its scene description role. Such a language should be generic 
> enough to also be suited for the shader description role, so it would be 
> rather pointless and user-unfriendly to devise yet another language for 
> that purpose.

OTOH the "shader" part has to, by necessity, be more limited than the
generic part of the language. It wouldn't make sense, for example, to
be able to create new objects into the scene while evaluating, for
example, the color of a reflected ray. That would mess up things quite
badly, I think.

(Also, the reason why shaders in graphics hardware can be so fast is
that they are highly parallelizable. Modern cards run thousands of
instances of shaders in parallel. Obviously there need to be some
limitations in the shader language for this to be possible. There
can't be inter-dependencies between shader instances being evaluated.)

-- 
                                                          - Warp

Post a reply to this message

Am 04.08.2015 um 16:10 schrieb Warp:
> clipka <ano### [at] anonymousorg> wrote:
>> Note that my goal is for the language /per se/ to have a very simple
>> syntax, and make it pretty much oblivious to rendering, except for
>> providing a few special native data types like 3D vectors and colours.
>> It'll be up to a bunch of predefined classes to fill it with raytracing
>> life in its scene description role. Such a language should be generic
>> enough to also be suited for the shader description role, so it would be
>> rather pointless and user-unfriendly to devise yet another language for
>> that purpose.
>
> OTOH the "shader" part has to, by necessity, be more limited than the
> generic part of the language. It wouldn't make sense, for example, to
> be able to create new objects into the scene while evaluating, for
> example, the color of a reflected ray. That would mess up things quite
> badly, I think.

That's part of the "fill it with raytracing life" thing: To add geometry 
to the scene, you'd invoke particular classes (such as a "sphere" class, 
a "box" class, and so on), which just won't be available in shaders.

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> Am 04.08.2015 um 16:10 schrieb Warp:
> > clipka <ano### [at] anonymousorg> wrote:
> >> Note that my goal is for the language /per se/ to have a very simple
> >> syntax, and make it pretty much oblivious to rendering, except for
> >> providing a few special native data types like 3D vectors and colours.
> >> It'll be up to a bunch of predefined classes to fill it with raytracing
> >> life in its scene description role. Such a language should be generic
> >> enough to also be suited for the shader description role, so it would be
> >> rather pointless and user-unfriendly to devise yet another language for
> >> that purpose.
> >
> > OTOH the "shader" part has to, by necessity, be more limited than the
> > generic part of the language. It wouldn't make sense, for example, to
> > be able to create new objects into the scene while evaluating, for
> > example, the color of a reflected ray. That would mess up things quite
> > badly, I think.
>
> That's part of the "fill it with raytracing life" thing: To add geometry
> to the scene, you'd invoke particular classes (such as a "sphere" class,
> a "box" class, and so on), which just won't be available in shaders.

why not simply add a shader option to the texture block, and use something like
OSL for the shader language?

http://code.google.com/p/openshadinglanguage/

Regards,
A.D.B.

Post a reply to this message

"Anthony D. Baye" <Sha### [at] spamnomorehotmailcom> wrote:
> clipka <ano### [at] anonymousorg> wrote:
> > Am 04.08.2015 um 16:10 schrieb Warp:
> > > clipka <ano### [at] anonymousorg> wrote:
> > >> Note that my goal is for the language /per se/ to have a very simple
> > >> syntax, and make it pretty much oblivious to rendering, except for
> > >> providing a few special native data types like 3D vectors and colours.
> > >> It'll be up to a bunch of predefined classes to fill it with raytracing
> > >> life in its scene description role. Such a language should be generic
> > >> enough to also be suited for the shader description role, so it would be
> > >> rather pointless and user-unfriendly to devise yet another language for
> > >> that purpose.
> > >
> > > OTOH the "shader" part has to, by necessity, be more limited than the
> > > generic part of the language. It wouldn't make sense, for example, to
> > > be able to create new objects into the scene while evaluating, for
> > > example, the color of a reflected ray. That would mess up things quite
> > > badly, I think.
> >
> > That's part of the "fill it with raytracing life" thing: To add geometry
> > to the scene, you'd invoke particular classes (such as a "sphere" class,
> > a "box" class, and so on), which just won't be available in shaders.
>
> why not simply add a shader option to the texture block, and use something like
> OSL for the shader language?
>
> http://code.google.com/p/openshadinglanguage/
>
> Regards,
> A.D.B.

Of course, using a shader with a patterned texture doesn't make a lot of sense,
so shaders could be used as alternatives to textures.

Then too, if we end up with an OO SDL, would the shader become an object?
Perhaps a subclass of texture... or maybe a superclass... sibling?

At any rate, my point is why make the job more difficult by designing two
complete languages, when we can use one that's already made.

Regards,
A.D.B.

Post a reply to this message

Am 05.08.2015 um 03:04 schrieb Anthony D. Baye:

>> why not simply add a shader option to the texture block, and use something like
>> OSL for the shader language?
>>
>> http://code.google.com/p/openshadinglanguage/

Uh... "simply"?

Maybe because we'd need to...

- design the new SDL in such a way that the OSL can be embedded therein 
without conflict;

- design the new SDL parser in such a way that the OSL can be plugged 
into it;

- design a layer that provides the exact features exposed to the user by 
the OSL;

- prepare for a never-ending struggle to explain to users why feature X 
is part of OSL but not supported by POV-Ray; and

- prepare for a never-ending struggle to explain to users why syntax Y 
in the SDL doesn't do the same as in the OSL.


Thanks, but no thanks.

/Maybe/ OSL will be supported by some POV-Ray 4.5, as an /alternative/ 
to the native shader definition syntax. But not in POV-Ray 4.0, and 
certainly not as the primary language for user-defined extensions.


> Then too, if we end up with an OO SDL, would the shader become an object?
> Perhaps a subclass of texture... or maybe a superclass... sibling?

Exactly that.


> At any rate, my point is why make the job more difficult by designing two
> complete languages, when we can use one that's already made.

Because (A) one of those two languages needs to be designed from scratch 
anyway, in order to be suitable as a Scene Description Language for 
users who don't want to be programmers.

I don't want users to have to write

      MySphere = new sphere;
      MySphere.center.x = 2.0;
      MySphere.center.y = 1.0;
      MySphere.center.z = 5.0;
      MySphere.radius = 0.5;
      MyScene.add(MySphere);

when a specially-designed language could make it a one-liner, like it is 
in the current SDL. I've yet to see a language other than SDL 3.x that 
fits the bill; but I know that a language can be designed to achieve 
that goal.

And (B) because the two complete languages will be just one. With a 
different set of objects to manipulate, but the same underlying language 
nonetheless.

Post a reply to this message

Anthony D. Baye <Sha### [at] spamnomorehotmailcom> wrote:
> why not simply add a shader option to the texture block, and use something like
> OSL for the shader language?

> http://code.google.com/p/openshadinglanguage/

The "shader" language would need to be able to do a lot more than just
get some values and input and return a color.

For example, it ought to be possible for it to say "shoot a ray starting
from this point towards this direction, and gimme all the info about
the object that the ray hits". (This kind of functionality could be
used, for instance, to create a "portal" shader. In other words, a
surface that shows a completely different part of the scene, as if
the suface were a "portal". Rather than this being a hardcoded feature,
the user ought to easily write such functionality as a "shader". As
such, it's then easy to create a myriad of variants with little effort.)

-- 
                                                          - Warp

Post a reply to this message

On Wed, 05 Aug 2015 19:29:42 +0200, Warp <war### [at] tagpovrayorg> wrote:

> Anthony D. Baye <Sha### [at] spamnomorehotmailcom> wrote:
>> why not simply add a shader option to the texture block, and use  
>> something like
>> OSL for the shader language?
>
>> http://code.google.com/p/openshadinglanguage/
>
> The "shader" language would need to be able to do a lot more than just
> get some values and input and return a color.
>
> For example, it ought to be possible for it to say "shoot a ray starting
> from this point towards this direction, and gimme all the info about
> the object that the ray hits". (This kind of functionality could be
> used, for instance, to create a "portal" shader. In other words, a
> surface that shows a completely different part of the scene, as if
> the suface were a "portal". Rather than this being a hardcoded feature,
> the user ought to easily write such functionality as a "shader". As
> such, it's then easy to create a myriad of variants with little effort.)
>

Ooo portals! I like portals =D

-- 
-Nekar Xenos-

Post a reply to this message

On 8/5/2015 11:29 AM, Warp wrote:
> For example, it ought to be possible for it to say "shoot a ray starting
> from this point towards this direction, and gimme all the info about
> the object that the ray hits". (This kind of functionality could be
> used, for instance, to create a "portal" shader. In other words, a
> surface that shows a completely different part of the scene, as if
> the suface were a "portal". Rather than this being a hardcoded feature,
> the user ought to easily write such functionality as a "shader". As
> such, it's then easy to create a myriad of variants with little effort.)
>

It would also allow all kinds of tricks involving translucency, 
reflections, etc. For instance, the example of blurred reflection you 
gave in the first post.

Post a reply to this message

"Anthony D. Baye" <Sha### [at] spamnomorehotmailcom> wrote:
> clipka <ano### [at] anonymousorg> wrote:
> > Am 04.08.2015 um 16:10 schrieb Warp:
> > > clipka <ano### [at] anonymousorg> wrote:
> > >> Note that my goal is for the language /per se/ to have a very simple
> > >> syntax, and make it pretty much oblivious to rendering, except for
> > >> providing a few special native data types like 3D vectors and colours.
> > >> It'll be up to a bunch of predefined classes to fill it with raytracing
> > >> life in its scene description role. Such a language should be generic
> > >> enough to also be suited for the shader description role, so it would be
> > >> rather pointless and user-unfriendly to devise yet another language for
> > >> that purpose.
> > >
> > > OTOH the "shader" part has to, by necessity, be more limited than the
> > > generic part of the language. It wouldn't make sense, for example, to
> > > be able to create new objects into the scene while evaluating, for
> > > example, the color of a reflected ray. That would mess up things quite
> > > badly, I think.
> >
> > That's part of the "fill it with raytracing life" thing: To add geometry
> > to the scene, you'd invoke particular classes (such as a "sphere" class,
> > a "box" class, and so on), which just won't be available in shaders.
>
> why not simply add a shader option to the texture block, and use something like
> OSL for the shader language?
>
> http://code.google.com/p/openshadinglanguage/
>
> Regards,
> A.D.B.


As a beginner, When I considered options of free renderers to learn, I chose
POV-Ray over Aqsis and other RIB like alternatives. One of the reasons was that
it had one language to learn instead of two, and this also felt a relief later
when deciding to script for an exporter, which had to be written in yet another
language (Python) (by the way Mitsuba has a full Python API/SDL which makes it
very attractive in this regard but it does miss a lot of the POV-Ray features)

Also I get the point that externalizing development by relying on solid
standards like the OSL you suggest allows developers to focus on more important
things, but the SDL is really central and one of the biggest selling point of
POV-Ray. It made me chose it not because it was standard, but because I trusted
it had the potential to be better than standard.

Post a reply to this message

On 7/30/2015 6:15 AM, Warp wrote:
> Some of these are things I have discussed years ago, but I thought it
> could be useful to revive the discussion a bit.
>
> One of the original plans for POV-Ray 4 was to make a complete rehaul
> of the SDL from scratch, making it a lot more versatilve, expressive,
> powerful and faster (via byte-compilation). This would have been a
> completely new input language that's different from the current one
> (but with, perhaps, a backwards compatibility mode to support existing
> scene files.)

I think basing things of SVG would be a good idea. I.e. XML + SMIL + 
JavaScript.


Mike

Post a reply to this message

Am 11.12.2015 um 01:20 schrieb Mike Horvath:
> On 7/30/2015 6:15 AM, Warp wrote:
>> Some of these are things I have discussed years ago, but I thought it
>> could be useful to revive the discussion a bit.
>>
>> One of the original plans for POV-Ray 4 was to make a complete rehaul
>> of the SDL from scratch, making it a lot more versatilve, expressive,
>> powerful and faster (via byte-compilation). This would have been a
>> completely new input language that's different from the current one
>> (but with, perhaps, a backwards compatibility mode to support existing
>> scene files.)
> 
> I think basing things of SVG would be a good idea. I.e. XML + SMIL +
> JavaScript.

Abso-bloody-lutely not.

XML is quite useful for the static description of purely hierarchical
structures comprised mainly of textual information; for anything else,
it is overly verbose and/or excessively difficult to employ.

A POV-Ray scene is indeed, in essence, a description of purely
hierarchical structures; however, that's where the applicability of XML
ends:

- The information in a POV-Ray scene is mostly non-textual, which means
that much of the overhead XML introduces is entirely unnecessary (such
as the enclosing of tags in angular brackets, or of attribute values in
double quotes).

- POV-Ray's current SDL excels (arguably not in speed, but certainly in
flexibility) in the dynamic generation of scenes; its description of
information is non-static, and a successor to its SDL should indeed
provide the same feature with similar ease. XML is poorly suited for
that: XML-based notations for control structures tend to be excessively
complicated.

The grafting of JavaScript on top of XML (or, more precisely, into its
bowels) does provide a way of generating information dynamically with
more ease than XML alone would allow; however, this is nothing more than
a makeshift solution -- one that has caught on, I give you that, but
that doesn't mean it's anywhere close to elegant: You do need to learn
two entirely different languages, XML and JavaScript.

In fact, the fusing of XML and JavaScript is as ugly a chimera as it can
possibly get: Some valid (and in fact pretty much essential) JavaScript
operators are invalid as XML content; as a matter of fact, /all/
characters that are invalid in XML content may occur as operators in
JavaScript. Thus, embedding arbitrary JavaScript into XML is only
possible via a workaround that hijacks XML comments - which should
normally be ignored - to contain relevant content. And even then, it is
possible to write JavaScript code snippets that break the mechanism by
which they are embedded into XML.

The only clean solution to this problem is to let the JavaScript
portions reside in entirely different files; but that defies the whole
purpose of having a unified language to describe content in a dynamic
fashion.


When it comes to data exchange formats, rather than XML, a much more
suitable candidate for a future POV-Ray SDL would be JSON. But even
then, I think it should only serve as a source for inspiration, rather
than being employed "out of the box".


In my opinion it remains a fact that the best language for POV-Ray would
be a domain-specific language (and by that I don't mean a
domain-specific abuse of an existing language like Apache Camel's DSL,
but a genuine language in its own right).

Post a reply to this message

In article <566ae5e5$1@news.povray.org>, ano### [at] anonymousorg says...
> > I think basing things of SVG would be a good idea. I.e. XML + SMIL +
> > JavaScript.
> 
> Abso-bloody-lutely not.

I thought the same thing. XML is a bitch to write by hand. Awesome for 
machine generation, though.

> You do need to learn
> two entirely different languages, XML and JavaScript.

Yeah, but povray is two fused languages now.  The DKBTrace "C" type of 
object definition (which I learned first...) and a perl-ish procedural 
language bolted on top of that.

I thought of JSON when he said XML, but is that really any better than 
what exists? It just replaces the current braces with brackets and 
braces.  One advantage might be, my only complaint about the current 
language is the inconsistent use of semicolons and commas.
#declare SemicolonRequired=1;
#debug "no semicolon here\n"
torus { 1, 0.1 }	// comma please
cone { <0,0,0> 1 <1,1,1> 2 }	// 4 params no commas

dik

Post a reply to this message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Le 11/12/2015 17:43, Dick Balaska a écrit :
> In article <566ae5e5$1@news.povray.org>, ano### [at] anonymousorg
> says...
>>> I think basing things of SVG would be a good idea. I.e. XML +
>>> SMIL + JavaScript.
>> 
>> Abso-bloody-lutely not.
> 
> I thought the same thing. XML is a bitch to write by hand. Awesome
> for machine generation, though.

writing once xml is not a problem... it's reading it later and trying
to amend it that is really painful.

> 
>> You do need to learn two entirely different languages, XML and
>> JavaScript.
> 
> Yeah, but povray is two fused languages now.  The DKBTrace "C" type
> of object definition (which I learned first...) and a perl-ish
> procedural language bolted on top of that.
> 
> I thought of JSON when he said XML, but is that really any better
> than what exists? It just replaces the current braces with brackets
> and braces.  One advantage might be, my only complaint about the
> current language is the inconsistent use of semicolons and commas. 
> #declare SemicolonRequired=1; #debug "no semicolon here\n" torus {
> 1, 0.1 }	// comma please cone { <0,0,0> 1 <1,1,1> 2 }	// 4 params
> no commas
> 
> dik
> 

If I was crazy, I would use C++ and a compiler as the parser, filling
a scene object with other objects... a rendering would be the
interaction of a camera with a scene. The only problem I would
foresee: the dynamic generation of code ... you know, the first part
of the parsing generate an include file to be included later... but do
we really need that ?(I'm afraid someone will want it).

Animation ? well, just a matter of looping...

Povray 3D objects, and all others aspects, just a library within a
dedicated namespace.

usual functions : just native coding.

Remaining problem: new users have to learn C++ and get a C++ compiler,
linking with the povray library.

Povray::Torus obj1(1,0.1);
// texturing can be a post constructor operation
Povray::Cone obj2( Povray::3DVector(0,0,0), 1, Povray::3DVector(), 2);
Povray::Scene take1;
take1 += obj1;
take1 += obj2;

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iJwEAQEIAAYFAlZrQhsACgkQhKAm8mTpkW0UCQP/Qe5KVy5Jm+Owwm8iCWRtcE1w
6rYxFlWFbmfHrsKzJLnZmbsyNPpYikd/cY1TU9mZ6JSZbuB19od+2XDs1wgYs+CF
SfADG628Gty79WXKueVPjdIv0q60jxNw8tDR+3GfjtJxMmN3xKGcZMn1rCnV1nYg
kkZ/flue97VAFzYnRmo=
=BKtc
-----END PGP SIGNATURE-----

Post a reply to this message

Am 11.12.2015 um 22:37 schrieb Le_Forgeron:

> If I was crazy, I would use C++ and a compiler as the parser, filling
> a scene object with other objects...

> Povray::Torus obj1(1,0.1);
> // texturing can be a post constructor operation
> Povray::Cone obj2( Povray::3DVector(0,0,0), 1, Povray::3DVector(), 2);

Still unnecessarily complex, as compared to (e.g.)

cone { <0,0,0>, <0,1,0>, 2 }


One of the key properties of a POV-Ray scene, which make the
requirements for the future SDL so unique, is that it may contain (A)
descriptions of complex algorithms as well as (B) descriptions of
hierarchical data, mixed arbitrarily.

XML, JSON & co. are useful for (B) but cumbersome for (A), while classic
programming languages excel at (A) but suck at (B).

(Add to this the fact that POV-Ray makes heavy use of vectors and
colours, which in most programming languages would have to be "bolted
on" artificially, which usually makes for less than ideal syntax.)


I re-iterate: The ideal SDL for POV-Ray has yet to be designed.

(POV-Ray 3.x SDL manages to cater to both requirements comparatively
well, but - as Dick correctly points out - sucks big time in terms of
consistency; also, it's somewhat outdated when it comes to describing
algorithms efficiently.)

I'm not writing this to crush people's hope for a new SDL; rather to the
contrary, I'd say. But I'd love to crush the hope of any language's
fanboys that their favorite language may become that new SDL, or even
form the basis of it, and -- maybe more importantly -- direct people
away from the (almost inevitably) fruitless search for an ideal existing
language, towards actually contributing to the task of defining a new
language from scratch (drawing some inspiration from other languages
where it seems fitting, but leaving well-trodden paths where they would
be inefficient).


One possible way of contributing to this process would be to simply
write down some hypothetical scene in whatever format you think a
next-generation POV-Ray SDL could look like. Some added text explaining
how that file format is structured would be nice to have, but could also
be deferred as a next step.


Remember, the format _must_:

- be efficient at describing static portions of a scene;

- be efficient at describing algorithms to construct portions of a scene
dynamically.

- be describing a scene in terms of essentially the same elements and
hierarchical structures as the current POV-Ray SDL (some individual
cleanup allowed, my favorite candidates there being the finish and
camera blocks);


As additional requirements, the format _must_:

- allow for two-step parsing, with the first step transforming the
verbose human-readable format into some kind of bytecode "program" that
is to be "executed" later by a virtual machine embedded into POV-Ray;
most notably this will prohibit "macros" from constructing only partial
statements, and dynamical generation of additional algorithmic portions
at execution time will probably be unavailable as well. (The idea behind
this requirement is to maximize parsing performance for scenes with
complex algorithms; the current plan calls for LLVM as the bytecode
format, allowing to even use just-in-time compilation of the scene file.)

- in its essential syntax be agnostic of the details of a scene's
elements and their hierarchy; e.g. the basic syntax must be unaware of
the fact that the element "texture" has a child element named "finish",
and instead only deal with the general fact that some elements have
child elements with specific names. (The idea behind this requirement is
manifold, from simplifying the parser to automatically enforcing more
consistency in the syntax, and even extending the exact same syntax to
user-defined elements.)


Also, the format _should_:

- support vectors and colours as native data types.

- provide some syntax to define custom geometric primitives, shading
models, and similar stuff.

Post a reply to this message

On 12/11/2015 6:32 PM, clipka wrote:
> One of the key properties of a POV-Ray scene, which make the
> requirements for the future SDL so unique, is that it may contain (A)
> descriptions of complex algorithms as well as (B) descriptions of
> hierarchical data, mixed arbitrarily.

One of the key properties that makes POV SDL great, but also one of the 
key properties that makes it unsupported by anyone else in the universe.

I don't like typing XML either but it might mean more third party tools 
in the future.


Mike

Post a reply to this message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Le 12/12/2015 00:32, clipka a écrit :
> Am 11.12.2015 um 22:37 schrieb Le_Forgeron:
> 
>> If I was crazy, I would use C++ and a compiler as the parser,
>> filling a scene object with other objects...
> 
>> Povray::Torus obj1(1,0.1); // texturing can be a post constructor
>> operation Povray::Cone obj2( Povray::3DVector(0,0,0), 1,
>> Povray::3DVector(), 2);
> 
> Still unnecessarily complex, as compared to (e.g.)
> 
> cone { <0,0,0>, <0,1,0>, 2 }
> 

Please notice, I wrote "If I was crazy".

of course C++11/14/17 could be abused with initialiser list in
constructor for:

scene += Cone( {0,0,0},{0,1,0}, 2);

that is just a matter of supported syntax for any language, I took
C++, but it would be the same with any other.
> 
> One of the key properties of a POV-Ray scene, which make the 
> requirements for the future SDL so unique, is that it may contain
> (A) descriptions of complex algorithms as well as (B) descriptions
> of hierarchical data, mixed arbitrarily.

[cutting the very good list of expectations]

Just beware of the NIH syndrome.
As well as over-generalisation (e.g. let make a super-generic system
to handle unthinkable evolutions), that's how you finish with heavy
XML inside SQL database, with mandatory manual joint using regular
expression applied on the single table of 2 or 3 columns, instead of
unfolding the whole data scheme across multiple dedicated tables (but
yes, a data scheme does not evolves to handle any extension, and due
to lack of initial strict requirements, you have a meta-engine instead
of a dedicated engine... so customers can customize it as they want...
well, there is no customer which are ready to pay for a meta-engine
and then pay again the customisation... excepted in the ERP/SAP
business model (or is it a mafia ?)).

Despite the C-like origin of SDL (or pascal... or whatever was fancy
in the 80's ), Povray is very object oriented.

Do we want to look at it like an interpreted or a compiled language,
it might be irrelevant for the speed, but in the approach provided to
the new users.
Do we want to write our own parser, or can it be left to an existing one
 ?

If it was one of boost library, would it still be povray ?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iJwEAQEIAAYFAlZr24EACgkQhKAm8mTpkW3eHgQAntLCC+Elu2b0LKan1I1Azonk
pPBuGjJLnWqJYwWBdtuPRzid7jA/sATpfVWrQ5MOZCFszDWU3t4Ao1HjhT73i2kP
bKFPs/P+bYWOd3mhnpgYDScOSs0aTL9UFZjY1jsr0La8aQf54ltl5iuyPbpbMm57
b+z5S/wR6ogjdEgu7mM=
=0HOR
-----END PGP SIGNATURE-----

Post a reply to this message

Am 12.12.2015 um 05:54 schrieb Mike Horvath:
> On 12/11/2015 6:32 PM, clipka wrote:
>> One of the key properties of a POV-Ray scene, which make the
>> requirements for the future SDL so unique, is that it may contain (A)
>> descriptions of complex algorithms as well as (B) descriptions of
>> hierarchical data, mixed arbitrarily.
> 
> One of the key properties that makes POV SDL great, but also one of the
> key properties that makes it unsupported by anyone else in the universe.

Actually that is only true when it comes to /import/ of POV-Ray scenes
into other software. As far as /export/ is concerned, I think the key
properties that make it unsupported by popular 3D software are that (a)
they're all entirely mesh-based, while POV-Ray isn't, and (b) work on
POV-Ray appeared to have stalled for a decade or so after the last 3.6
release.

As a matter of fact, outside the mesh-based world, POV-Ray is still
comparatively popular as an export format.

> I don't like typing XML either but it might mean more third party tools
> in the future.

Choosing XML as the basis for an SDL would (or should) not change the
aforementioned key property of a POV-Ray scene, the mixing of static
data and algorithms, which you claim to be in the way of broader support.

One road I would be willing to go along would be to define a static XML-
(or better yet JSON-) based format, devoid of any scripting elements,
intended solely for efficient data exchange with other programs, to be
supported for input as an /alternative/ to the official SDL intended for
manual editing; also, one mode of operation for POV-Ray could be to not
actually render a scene, but rather to just convert it to that data
exchange format, effectively constituting data export.

Also, for improved integration into the mesh-based world, I think
winning features would be (A) inbuilt support for .obj as a mesh input
format, and (B) inbuilt support for tesselation and .obj export.

Another way to improve support from 3rd party tools could be to provide
the parser and tesselation code as library modules, to serve as a basis
for 3rd party tool import modules.

Post a reply to this message

Am 12.12.2015 um 09:32 schrieb Le_Forgeron:

> [cutting the very good list of expectations]
> 
> Just beware of the NIH syndrome.

There's really nothing to beware there; if I could find a language out
there that does suit POV-Ray's needs, I'd probably be the first to go
for it; if nothing else, it would save us all the work of defining the
new SDL in the first place; a pre-existing parser to just plug into
POV-Ray would save a lot of work, too.

Also, if a few modifications to an existing language would get us there
(e.g. bolting some additional syntax for vector and colour data types to
the language), that's what I'd do.

But I won't sacrifice what I consider essential requirements, just to be
able to re-use existing components.


> Do we want to look at it like an interpreted or a compiled language,
> it might be irrelevant for the speed, but in the approach provided to
> the new users.

"We" as the developers definitely want to look /at it/ like a compiled
language, because that's what will give us the best performance in the
parsing of complex scenes.

As for whether the language is to /look like/ an interpreted or compiled
language to the user, you tell me what the difference is and I tell you
what we want. Except for dynamic creation of executable code at
execution time, I can think of nothing that would clearly delineate
interpreted languages from compiled ones.

If you ask whether we want a strongly or weakly typed language, then I'd
say that we want the language to generally be weakly typed, because
that's easier to use; but we probably also want strongly typed elements,
so that we get an early failure if someone mistypes a property name of
e.g. a geometric primitive, or tries to use e.g. a geometric primitive
where a texture is expected.

> Do we want to write our own parser, or can it be left to an existing one
>  ?

We certainly don't /want/ to write our own parser, because we have
enough other work to do, but if we have to (because the language does
not exist yet, or there's no open source parser for it yet, or no
embedded parser targeting an existing portable VM, or none that is
modular enough to be used as such), that's what we /will/ do out of
necessity.


> If it was one of boost library, would it still be povray ?

If boost contained a library providing a suitable embedded language, I'm
sure we'd already be halfway through integrating it into POV-Ray.

Post a reply to this message

On 12/11/2015 01:20 AM, Mike Horvath a dit:

> I think basing things of SVG would be a good idea. I.e. XML + SMIL +
> JavaScript.

    XML is just a giant garbage.

-- 
http://foo.bar.quux.over-blog.com/article-thsf-2013-114632120.html

Post a reply to this message

Le 15-12-13 09:30, tth a écrit :
> On 12/11/2015 01:20 AM, Mike Horvath a dit:
>
>> I think basing things of SVG would be a good idea. I.e. XML + SMIL +
>> JavaScript.
>
>     XML is just a giant garbage.
>

I think that you may over-react to some degree here.
XML does have some very legitimate uses, the real problem, is that's it 
have been abused in ways that where unintended to do things that are not 
supposed to be supported.
The base is mostly OK.
The abuse IS garbage.

Post a reply to this message

> Please notice, I wrote "If I was crazy".
>
> of course C++11/14/17 could be abused with initialiser list in
> constructor for:
>
> scene += Cone( {0,0,0},{0,1,0}, 2);
>
> that is just a matter of supported syntax for any language, I took
> C++, but it would be the same with any other.

I've got a good scene:

scene += Cone( {0,0,0},{0,1,0}, 2);
std::remove("some important file");

Is there any way to limit what damage a compiled C++ program can do?

Post a reply to this message

Am 14.12.2015 um 09:41 schrieb scott:
>> Please notice, I wrote "If I was crazy".
>>
>> of course C++11/14/17 could be abused with initialiser list in
>> constructor for:
>>
>> scene += Cone( {0,0,0},{0,1,0}, 2);
>>
>> that is just a matter of supported syntax for any language, I took
>> C++, but it would be the same with any other.
> 
> I've got a good scene:
> 
> scene += Cone( {0,0,0},{0,1,0}, 2);
> std::remove("some important file");
> 
> Is there any way to limit what damage a compiled C++ program can do?

Absolutely.

If by "compiled" you really do mean "compiled", it's as easy as
permitting only linkage to a tightly controlled set of libraries, of
which you know that they can't possibly do any harm (which may require
providing a custom runtime library).

If by "compiled" you actually mean "built", i.e. compiled /and/ linked,
it's as "easy" as running the program in a virtual machine.

Post a reply to this message

>> I've got a good scene:
>>
>> scene += Cone( {0,0,0},{0,1,0}, 2);
>> std::remove("some important file");
>>
>> Is there any way to limit what damage a compiled C++ program can do?
>
> Absolutely.
>
> If by "compiled" you really do mean "compiled", it's as easy as
> permitting only linkage to a tightly controlled set of libraries, of
> which you know that they can't possibly do any harm (which may require
> providing a custom runtime library).

That sounds a lot of work, with still a small risk of someone finding a 
loophole to do something bad in their scene.

> If by "compiled" you actually mean "built", i.e. compiled /and/ linked,
> it's as "easy" as running the program in a virtual machine.

Sounds a lot of work too, so that it's transparent to the casual user 
that the scene is being rendered in a virtual machine.

Post a reply to this message

scott <sco### [at] scottcom> wrote:
> >> I've got a good scene:
> >>
> >> scene += Cone( {0,0,0},{0,1,0}, 2);
> >> std::remove("some important file");
> >>
> >> Is there any way to limit what damage a compiled C++ program can do?
> >
> > Absolutely.
> >
> > If by "compiled" you really do mean "compiled", it's as easy as
> > permitting only linkage to a tightly controlled set of libraries, of
> > which you know that they can't possibly do any harm (which may require
> > providing a custom runtime library).
>
> That sounds a lot of work, with still a small risk of someone finding a
> loophole to do something bad in their scene.
>
> > If by "compiled" you actually mean "built", i.e. compiled /and/ linked,
> > it's as "easy" as running the program in a virtual machine.
>
> Sounds a lot of work too, so that it's transparent to the casual user
> that the scene is being rendered in a virtual machine.

I didn't say there's any /feasible/ way (nor did I mean to) - just that there
/is/ a way ;)

Post a reply to this message

Am 12.12.2015 um 13:27 schrieb clipka:
> Am 12.12.2015 um 09:32 schrieb Le_Forgeron:
> 
>> [cutting the very good list of expectations]
>>
>> Just beware of the NIH syndrome.
> 
> There's really nothing to beware there; if I could find a language out
> there that does suit POV-Ray's needs, I'd probably be the first to go
> for it; if nothing else, it would save us all the work of defining the
> new SDL in the first place; a pre-existing parser to just plug into
> POV-Ray would save a lot of work, too.

I /think/ I've found a language that might be suitable; at any rate, it...

(1) shares the design goal of providing both declarative and procedural
elements;

(2) has native support for /all/ of the domain-specific basic types we
would need (most notably vectors and colours);

(3) already covers a major subset of POV-Ray's language domain; and

(4) is already established, /specifically/ in the domain of 3D rendering.


Have a look at NVIDIA's Material Definition Language (MDL):

http://www.nvidia-arc.com/fileadmin/user_upload/iray_documentation/nvidia_mdl_introduction.140512.A4.pdf

http://www.nvidia-arc.com/fileadmin/user_upload/MDL_spec_1.2.2_28Apr2015.pdf

I think all we'd have to do is provide some intrinsic data types for
geometric shapes and some odds & ends (such as light sources, global
settings, etc.), and we'd have a useful Scene Description Language.

Is it pretty to use? Don't know yet.

Can we get Nvidia's implementation for free? Don't know yet, but I'd
doubt it. The language specs are open though, so we could write our own
parser.

Post a reply to this message

Le_Forgeron <jgr### [at] freefr> wrote:
> If I was crazy, I would use C++ and a compiler as the parser

This is actually not such an absolutely crazy idea. Consider that
Apple's Metal shading language (which is an alternative to glsl)
uses C++11 as its syntax (and AFAIK pretty much a C++11 compiler
to generate the shader binary). These shaders are compiled at
runtime rather than compile time (they are not compiled from
*within* the shader code, but from the calling code, but
nevertheless, it's done at runtime.)

Nowadays this would be extremely feasible because LLVM has been
designed from the ground up for precisely this kind of thing.

-- 
                                                          - Warp

Post a reply to this message

Le_Forgeron <jgr### [at] freefr> wrote:
> > Still unnecessarily complex, as compared to (e.g.)
> > 
> > cone { <0,0,0>, <0,1,0>, 2 }
> > 

> Please notice, I wrote "If I was crazy".

> of course C++11/14/17 could be abused with initialiser list in
> constructor for:

> scene += Cone( {0,0,0},{0,1,0}, 2);

Actually I don't see a reason why this line of code:

    cone( { 0, 0, 0 }, { 0, 1, 0 }, 2);

couldn't add the cone to the (default) scene.

It could even work something like this:

    Cone( { 0, 0, 0 }, { 0, 1, 0 }, 2)
    .pigment( some pigment def here )
    .finish( some finish def here )
    .rotate(10, 20, 30)
    .scale(0.5, 1.0, 2.0);

-- 
                                                          - Warp

Post a reply to this message

Am 23.05.2016 um 18:02 schrieb Warp:
> Le_Forgeron <jgr### [at] freefr> wrote:
>> If I was crazy, I would use C++ and a compiler as the parser
> 
> This is actually not such an absolutely crazy idea. Consider that
> Apple's Metal shading language (which is an alternative to glsl)
> uses C++11 as its syntax (and AFAIK pretty much a C++11 compiler
> to generate the shader binary). These shaders are compiled at
> runtime rather than compile time (they are not compiled from
> *within* the shader code, but from the calling code, but
> nevertheless, it's done at runtime.)
> 
> Nowadays this would be extremely feasible because LLVM has been
> designed from the ground up for precisely this kind of thing.

Not specifically for C++, but yes.

Use of LLVM in a future POV-Ray SDL parser can be considered a given,
both for the scene in general and, even more importantly, any
user-defined "functions" (which will presumably be much more powerful
than they are now).

(Unless something even better than LLVM happens to pop up before we
actually start working on that new parser.)

Post a reply to this message