In article <cjameshuff-C93C28.15390704012005@news.povray.org>, 
cja### [at] earthlinknet says...
> In article <41d86f9a$1@news.povray.org>, Darren New <dne### [at] sanrrcom> 
> wrote:
> 
> > Once you get to the point of *interpreting* what "#while" means, you're 
> > past the parsing stage. POV-Ray may call that "parsing", but it isn't. 
> > It's executing the specification. In other words, if the OP wants to do 
> > something like add a namespace to some selected number of variables, you 
> > don't have to understand a "while" loop to find all the variables you're 
> > going to change, any more than emacs needs to understand SDL to 
> > search-and-replace spheres with something else.
> 
> When POV says it's parsing, it's really parsing. POV executes the scene 
> file as it parses it, rather than parsing into an intermediate form to 
> be executed later. This is one of the reasons it is so slow, and one 
> thing which badly needs improvement in POV4...it crawls around the input 
> source files while generating the scene. The body of a loop is actually 
> parsed once per iteration. Then there's a postprocessing stage where 
> stuff like photons and radiosity are precalculated, basically "filling 
> in the blanks" of the generated scene, and then you get to the rendering 
> stage.
> 
Ah hah... I knew I was right about that. I look forward to a more JIT 
type system in 4.0, if they do so.


-- 
void main () {

    call functional_code()
  else
    call crash_windows();
}

Post a reply to this message

In article <41db0035@news.povray.org>, Sil### [at] gmxde says...
> I don't understand that one. Why should people _need_ to learn new 
> languages. If there is just a frontend that is able to create SDL code 
> they'd still have SDL code to work with. They won't be able to 
> effectively modify it but they aren't now either in many cases (those 
> that are just to hard to be treated with common SDL).
> 
Except that the frontend is entirely unneeded if you extend the SDL to 
support things properly in the first place. Moray does a decent, but not 
complete job of acting as a front end and even solves some problems, in a 
general way, like animation that are troublesome with pure SDL. But it 
falls short of supporting some basic things, like being able to link a 
light source to an object, so they move together, let alone some more 
complex things. The more abstraction you put between the core functions 
and the user, the more limited you end up making their options and the 
more the perception becomes, "That program can't do X". Of course, some 
times, like with media for subsurface scattering, the 'correct' solution 
can actually have the opposite result, making it so complicated to use 
the feature that the abstracted simulation of it might be preferable, but 
that is generally a far more rare situation. Any abstraction shouldn't be 
in the language someone uses to manually edit things in most cases, but 
more like Moray, when the abstraction is to a simpler representation of 
what it really being done. XML or other structures can't improve the 
ability to see that, it only obfuscates the actual process or nature of 
the objects in it even more.

-- 
void main () {

    call functional_code()
  else
    call crash_windows();
}

Post a reply to this message

In article <41dc85a1$1@news.povray.org>, Darren New <dne### [at] sanrrcom> 
wrote:

> > And, if it's not designed for it from the start, making the VM 
> > high-performance for things like shaders would probably be quite 
> > difficult.
> 
> Errr, well, I'd expect there's be a call that says "Trace." :-) Tcl's 
> all in C, and it's designed to call C routines pretty seemlessly. I 
> wouldn't think you'd implement the shaders in Tcl, but in C.

C is impractical for other reasons...you'd need a C compiler, obviously. 
This is a bigger problem than you'd think. The compiler must exist for 
all the platforms POV is likely to run on, and is an obstacle to porting 
to new platforms.

Then you need to run this external compiler with the shader code and 
link in the result. A C development environment is a bit heavy to 
include along with the raytracer, and is likely to be a huge source of 
problems on different configurations. Plus there's the security risks of 
using such a low level language...you'd pretty much have to disable 
programmable shaders for a web-driven application, for example.

Also, with no support for vector/color math or operator overloading, C 
shaders would get pretty ugly...though if you can solve the compiler 
problem, I suppose you could translate from a customized language into 
C, as was done for early C++ and Objective C compilers.

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: <chr### [at] tagpovrayorg>
http://tag.povray.org/

Post a reply to this message

In article <41dcb8a3@news.povray.org>, Darren New <dne### [at] sanrrcom> 
wrote:

> > C is impractical for other reasons...you'd need a C compiler, obviously. 
> 
> Um, well, yes. I was under the impression POV-Ray is written in C 
> already anyway. Indeed, you'd need a C compiler to extend POV-Ray in C. 
> If your implication is that no code written in C can be run on a user's 
> machine that doesn't have a C compiler installed... erm... :-)

No. I'm saying you'll need a compiler to run C source code, such as 
shaders written in C.


> I suspect Tcl already runs on more platforms than POV-Ray, altho I 
> wouldn't guarantee it runs on every platform POV-Ray runs on. I suspect 
> Tcl runs on more platforms than you think it does, tho. ;-)

I'm talking about the C compiler you suggested using for shaders. GCC 
might suffice, if the license allows bundling it like this, but it would 
add several megabytes at minimum to the POV-Ray distribution.


> You'd have to compile it, once per platform. Just like any attribute 
> you'd add to the SDL anyway. You wouldn't need to recompile the base 
> POV-Ray, tho.

But you would still have to compile the shader. It would probably then 
be loaded as a dynamic library by the POV executable, assuming the 
interface for doing so is reasonably easy to port. Tcl might have 
built-in facilities for doing this portably, but it's the first step 
that's hard.


> > A C development environment is a bit heavy to 
> > include along with the raytracer, and is likely to be a huge source of 
> > problems on different configurations.
> 
> I don't need a C development environment to run POV-Ray. I don't 
> understand what point you're trying to make.

Because it has its own custom language for functions, the closest thing 
it currently has to shaders. If you used C for a shader language, you 
would need a C development environment.


> Yes. Except that's already built into Tcl. It's called a "safe 
> interpreter". Been around for about 15 years now. I work with the guys 
> who invented it.

I don't see how that helps the security of the code written in C.


> Well, it's possible I just don't understand what you mean by a "shader". 
> However, what I was *trying* to say was, you can extend the SDL with C 
> if the extension is compute-intensive, and it wouldn't need to be part 
> of the official POV-Ray release, but it wouldn't be an entirely new 
> program either - just an extention.

That's essentially what a shader is. A mini-program used to do texturing 
and lighting calculations for the renderer. (Sometimes other stuff like 
geometry as well.)
They're typically compiled to bytecodes and interpreted. I think the big 
packages do let you load C object code or libraries, but you need the 
development tools...


> Or you can extend it directly in SDL, if the performance is good enough. 
> You could, for example, write Tcl code to import Wings3D files directly. 
> If that was too slow, you'd build an ImportWings.DLL (an ImportWings.SO 
> and etc) to run faster, and write Tcl code that if the DLL isn't there, 
> you fall back to the slower method.

But if Tcl doesn't even have built-in numeric calculations, it would 
probably be difficult to add them in a way that gets good enough 
performance for shaders.

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: <chr### [at] tagpovrayorg>
http://tag.povray.org/

Post a reply to this message

On Wed, 05 Jan 2005 21:35:34 -0800, Darren New <dne### [at] sanrrcom> wrote:
> > But you would still have to compile the shader.
>
> I'm not following. Why would I need to compile it myself, any more than 
> I need to compile POV-Ray myself? Why would compiling a shader be harder 
> than compiling POV-Ray is today?

I have a feeling that what Christoph is referencing here is that current
functions engine can be compiled into so called machine code without any
external C/CPP compiler. AFAIK all is required is kind of table for
"translating" function virtual machine codes into cpu codes as (IIRC) it is
done for (some of) Macs.

4.5.1.1.2 Fast Functions
http://www.povray.org/documentation/view/3.6.1/747/

ABX

Post a reply to this message

In article <l6spt0lpe76t20v524bkp5naifi490qfka@4ax.com>,
 ABX <abx### [at] abxartpl> wrote:

> I have a feeling that what Christoph is referencing here is that current
> functions engine can be compiled into so called machine code without any
> external C/CPP compiler. AFAIK all is required is kind of table for
> "translating" function virtual machine codes into cpu codes as (IIRC) it is
> done for (some of) Macs.
> 
> 4.5.1.1.2 Fast Functions
> http://www.povray.org/documentation/view/3.6.1/747/

No, that's not what I'm referring to. I do know where the 
miscommunication is happening though...see other reply.

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: <chr### [at] tagpovrayorg>
http://tag.povray.org/

Post a reply to this message

On Thu, 06 Jan 2005 06:44:34 -0800, Darren New <dne### [at] sanrrcom> wrote:
> So why would you need a C compiler to do this? That's what I'm not 
> following. Why is it easier to do that on POV-Ray 3.6 than it would be 
> to do on POV-Ray 4.0? Why not just do it the same way as already works?

I do not know what was meant in earlier discussion here but I suppose that
build-in compiler is in POV-Ray 3.6 because POV-Ray 3.6 is between POV-Ray 3
and POV-Ray 4. In other words, I think that's imposssible to introduce all the
changes in single release without releasing anything for years.

ABX

Post a reply to this message

In article <41dcce25$1@news.povray.org>, Darren New <dne### [at] sanrrcom> 
wrote:

> I don't need a C compiler to run shaders written in C, any more than I 
> need a C compiler to run POV-Ray written in C.

You do, since shaders are almost never distributed as native machine 
code.


> > I'm talking about the C compiler you suggested using for shaders.  GCC
> > might suffice, if the license allows bundling it like this, but it would 
> > add several megabytes at minimum to the POV-Ray distribution.
> 
> I'm not sure why you wouldn't distribute shaders (or whatever you wanted 
> efficiently) in the same way you distribute POV-Ray now.

Because shaders are written by the users, not the developers. This is 
the point I think you've been missing. Shaders are used when the built 
in primitives are too limited or awkward to accomplish a given job.


> > Because it has its own custom language for functions, the closest thing 
> > it currently has to shaders.
> 
> This has too many typos and pronouns for me to figure out what you're 
> talking about. What's "it"?

"It" is POV-Ray. It supports user-defined functions which are compiled 
into opcodes for a virtual machine, and optional further compilation 
into native machine code. These functions can be used to define 
isosurfaces, patterns, and with a little extra work, pigments. Not quite 
a full-blown shader language, but the closest thing POV-Ray currently 
has.


> > If you used C for a shader language, you 
> > would need a C development environment.
> 
> I'm not sure what a shader language is. I'm not suggesting a shader 
> language, but rather the use of Tcl as the basis for SDL.

Rather than assembling a material out of predefined patterns, the user 
writes a "shader program", a function that computes the resulting color 
for the renderer. Tcl is probably quite unsuitable for this, so you 
would need a separate language just for shaders and similar 
high-performance functions. There isn't any particular reason the entire 
SDL couldn't use the same VM. Some capabilities, such as defining 
objects, would be "low performance" and would be a bad thing to put in a 
shader, but I don't see a definite reason to have separate languages.


> > I don't see how that helps the security of the code written in C.
> 
> I don't know why you're fixated on C.

Because you suggested it as a high-performance alternative to Tcl, for a 
shader language.


> > They're typically compiled to bytecodes and interpreted.
> 
> You mean, like Tcl.

Superficially. What I was trying to point out was the difference from C, 
in that machine-specific forms are rarely distributed widely. You need a 
compiler that supports the platforms, or you need a VM capable of high 
performance FP math. That practically means a custom VM, 


> > But if Tcl doesn't even have built-in numeric calculations, 
> 
> Of course it has numeric calculations built in. I just said the syntax 
> isn't quite as standard as you might want, nor are numeric expressions 
> likely as efficient as a language compiled all the way down to machine code.

Earlier, you said:
> Mind, Tcl isn't really good at math as such, but on the other hand since 
> math isn't technically built into the language, that would probably be 
> pretty easy to improve.

I might have misunderstood this, but I took it to mean that math is 
implemented on top of mechanisms provided by the VM, rather than built 
into the VM. The extra abstraction can be useful, but can really hurt 
performance in floating-point heavy stuff like shaders.

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: <chr### [at] tagpovrayorg>
http://tag.povray.org/

Post a reply to this message

In article <41dd6fb2$1@news.povray.org>, Darren New <dne### [at] sanrrcom> 
wrote:

> Let's reset a little bit. I'm talking about whether it would be better 
> to base SDL 4.0 on an existing extensible scripting language with 
> ray-tracing primitves added, rather than basing it on a whole new 
> parser/runtime.  (Note: I am not suggesting anything about POV4. Just 
> talking. :-)
> 
> In either case, if the primitves are inadequate and you don't have a 
> precompiled version of the shader, then you're SOL. So maybe I'm missing 
> why shaders even came up in the discussion.

It's a reason why a new parser/runtime could be better than an existing 
one, like your suggestion of Tcl.


> > "It" is POV-Ray. It supports user-defined functions which are compiled 
> > into opcodes for a virtual machine, and optional further compilation 
> > into native machine code. These functions can be used to define 
> > isosurfaces, patterns, and with a little extra work, pigments. Not quite 
> > a full-blown shader language, but the closest thing POV-Ray currently 
> > has.
> 
> OK. And surprisingly enough, this implementation doesn't require a C 
> development environment. So....

POV doesn't currently use a "standard" programming/scripting language 
like Tcl. It uses its own language, and its own VM, designed 
specifically for floating point functions. Isosurfaces would render a 
lot slower if their functions were written in Tcl. They'd be faster if 
they were written in C, but that isn't very practical.


> Actually, I'd expect it to be quite suitable, but relatively slow since 
> it is after all byte-compiled. If you wanted it to be fast, you'd either 
> do something like POV-Ray does now, or you'd write the shader primitve 
> you're interested in in a faster compiled language like C and 
> dynamically load it as needed, or you'd write it in Tcl with possibly an 
> option to look for the already-compiled code and load that if it's 
> available.

We're talking about something that could be a complex function evaluated 
for every ray traced, possibly hundreds of times per pixel...thousands 
of times with antialiasing. Much more with blurred reflection, media 
shaders, and so on, and there's no reason the intersection calculations 
themselves couldn't be done in it for custom shapes. "Relatively slow" 
is too slow. Interpreted languages are barely suitable but practically 
necessary, which is why the POV-Ray function VM has hooks for JIT 
compilation. You really need a language optimized for floating point 
calculations, preferably with optimizations for colors, points, and 
normals.


> The benefit having to do with compiled shaders is the same, regardless 
> of what syntax you use for the SDL. Either you support user-defined 
> shaders in some custom language, or you support them as C libraries, or 
> you don't support them at all, and all three of those are available 
> regardless of the syntax of the language.

But once you have a custom language for shaders, why do you need another 
language like Tcl for the SDL? You could use one language, one VM for 
both purposes. The result is less internal code, and less stuff for 
people to learn.


> > Superficially. What I was trying to point out was the difference from C, 
> > in that machine-specific forms are rarely distributed widely.
> 
> Well, ask yourself why.  It's not like *no* software is widely 
> distributed precompiled. If it's easy to dynamically load shaders on 
> demand, there's no reason they couldn't be precompiled. Imagine an SDL 
> that would do a google search for a precompiled shader for your platform 
> if it ran into a Tcl implementation of that shader....

They can't be precompiled because they are written and distributed by 
users! You can't expect every user that wants to write shaders for 
others to use to have a C development environment with cross-compilers 
for every platform someone might want to use it on.

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: <chr### [at] tagpovrayorg>
http://tag.povray.org/

Post a reply to this message

In article <41ddac99@news.povray.org>, Darren New <dne### [at] sanrrcom> 
wrote:

> Translating the description of the shader into specific machine code or 
> specific byte codes would be the same amount of work to implement 
> regardless of whether you used Tcl or not to invoke the scripts.

Writing and maintaining both a shader language + VM and a Tcl SDL would 
be more work than writing and maintaining an SDL that also serves as a 
shader language. If you have a decent custom language already in the 
project, I see no reason to add another one.


> I expect the way you would use Tcl in this project is to implement 
> "sphere" in Tcl to output the appropriate bytecodes for the optimized 
> VM. I don't see writing the ray-tracer itself (i.e., the thing that 
> follows the rays and calculates pixel colors) in Tcl to be feasible any 
> more than doing so in SDL is feasible.

But maybe you don't want a plain sphere. Maybe you want a sphere with 
ripples on it. Thus, you need to execute user-defined code, and the 
faster the better.
With an improved custom-purpose SDL, replacing most of the raytracer 
with user-written code would be slower, but perfectly feasible. You 
could even implement your own raytracer, though there would be no reason 
I can think of to do so.


> Basically, you'd use Tcl to build the scene, and then your specific VM 
> to interpret the stream.  Layering, ya know?

But what purpose is there to another layer?


> The language that builds the VM byte stream doesn't have to be 
> especially fast. I mean, heck, POV-Ray 3.x (as far as I understand) 
> actually reparses the source text to do so. Tcl would be faster because 
> after the first time thru the loop, the parsing is byte-compiled.

But not fast enough. Not all VMs are equal. It's byte compiled, so are 
POV functions. However, the POV function VM is built for crunching 
numbers, and is almost certainly much faster at it than Tcl.


> > But once you have a custom language for shaders, why do you need another 
> > language like Tcl for the SDL?
> 
> The "custom language" would be Tcl.

No, the custom language I'm speaking of here is the shader language. 
Tcl's too slow for that.


> > You could use one language, one VM for both purposes.
> 
> Well, are you assuming that a VM needs a custom language to write a 
> compiler for? That may be the miscommunication.

No, of course not. The "custom language" could be essentially a clone of 
Tcl, with floating point and vector optimizations. Then there'd be even 
less reason to use standard Tcl.


> You can expect, however, that there is at least one person who has a 
> compiler for your platform, or POV-Ray wouldn't be running there. The 
> stumbling block isn't that compiled versions don't work for some reason. 

Yes, it is! I can't run x86 machine code on my iBook. Without a 
compiler, I can't run stuff I just wrote in C. Having someone else 
compile stuff for me every time I tweak the code just isn't an option.


> The stumbling block is a lack of distribution facilities where you can 
> find it precompiled. If there's a central place holding shaders, there's 
> a central place where people could contribute executable versions.

No, it isn't. I can't go download a compiled version of a shader I just 
wrote. Again, shaders are written by *users*. They need to be able to 
write the code, then run it. It needs to run as fast as feasible.


> Plus, if the shaders are written to compile down to some VM, the VM 
> code could be either built into POV or could be distributed along 
> with or instead of the source.

That is exactly what is done with many shaders, though it is a bit more 
flexible and useful to distribute the source.


> I mean, Tcl has a whole bunch of executable extensions that are written 
> and distributed by users. It's really not that hard. :-)  And I suspect 
> that POV-Ray doesn't run on *so* many platforms that, say, 6 different 
> executables wouldn't satisfy >95% of the users using it, and the other 
> 5% probably have the development environment anyway.

Requiring people to have a 6-way cross-compiling development environment 
in order to write and distribute a reasonably advanced scene is 
ridiculous. Alternatively, requiring them to have a development 
environment for their own system or know someone who does in order to 
render the scene is equally ridiculous. Interpreted languages are 
practically necessary for useable shader languages. However, even the 
fastest ones are also nearly too slow to be of practical use for that.

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: <chr### [at] tagpovrayorg>
http://tag.povray.org/

Post a reply to this message