Lets start commenting on the discussion.


Rudy Velthuis wrote:
> 
> Spider schrieb in Nachricht <36B7317E.4D09F18E@bahnhof.se>...
> >
> >
> >Rudy Velthuis wrote:
> 
> >> But as the POV-team already mentioned somewhere, most time is not wasted
> in
> >> parsing, but in allocating objects during this process. If this is true
> (I
> >> can't verify it), the pre-compile wouldn't do a lot of good.
> >It depends on the scene, as far as I know. in some cases, I've spent a
> >lot of time waiting for a parse to finish. (take a look at the recursive
> >tree creation algorithms. Those are very long parsing, but not long if
> >declared as a .inc. (try the WinTrees if you like:-) it is from this
> >experience that I came with the idea.
> 
> I'll have to take a look at the parser, but if this is true, then it's
> definitely not the allocation, which takes time, as obviously the result
> (the number of objects) would be exactly the same, so the number of
> allocations would be the same too.
Yes.
 
> Only difference is the fact, that WinTree does the recursion once (in Visual
> Basic), 
*GASP*
Whatta heck are you accusing me of?
It is Delphi, admittedly using a very sloppy design, and rxlib, and
dElphi 3, but... "VISUAL BASIC" ????! *gasp* *choke*

> producing an include file with all objects already spelled out, and
> the recursive parser does this while parsing. So parsing is definitely the
> culprit here, not allocation.
Yes, it is. That's why I made the prog. the allocation takes around 4-8
seconds, depending on the size of the tree.
 
<snip>
> 
> Perhaps there would be demand for a small program, which unravels #while
> loops and macros and creates (huge) include files out of them, taking a bit
> out of the parsing POV-Ray does. This could work like WinTree, but perhaps a
> bit more generalized. So the program would turn a:
> 
>   #declare Count = 0;
>   #while (Count <= 40)
>     sphere { <-1, Count, -1>, 1 }
>     #declare Count = Count + 1;
>   #end
> 
> into:
>   // #declare Count = 0;
>   // #while (Count <= 40)
>   sphere { <-1, 0, -1>, 1 }
>   sphere { <-1, 1, -1>, 1 }
>   sphere { <-1, 2, -1>, 1 }
>   ...
>   // #declare Count = Count + 1;
>   // #end
>   #declare Count = 40;
yes, this might be helpful, but perhaps not good enough. This is a thing
too look deper into.

> etc. (40 sphere definitions instead of one simple loop, and Count = 40, we
> might perhaps need this later on).
yes.

I'll go on commenting now :-)

//Spider

Post a reply to this message

<snippety>

> I do wonder, though, if POV could be coaxed to output a .pov file
> in canonical form as it parses a set of .pov and .inc files - with
> commas inserted where necessary and looping and conditionals and
> symbols and includes (all the #-stuff) and comments removed.  I
> think this goes a bit further than you were proposing.  It'd be
> less readable to humans, but it would be oh-so-easy for third-party
> parsers to read, and perhaps slightly faster for POV as well.  Some
> files, like the mandelbrot set made of spheres, would run much more
> quickly.
This is an interesting idea, but, it would cost some problems in the
final stages, since the main idea was that the parse would happen once
for more time consuming objects, as the tree or the mandelbrot you
mention. then one could spend several tries moving lights so the shadow
goes "just right" and so on. As well as testing all the textures one had
in mind for it...

The idea with a general pseudo pov might be ok, but then, might not...

//Spider

Post a reply to this message

Rudy Velthuis wrote:
> 
> Ron Parker schrieb in Nachricht <36b8526e.0@news.povray.org>...
> >On Wed, 3 Feb 1999 14:10:20 +0100, Rudy Velthuis <rve### [at] gmxnet> wrote:
> >>I have read somewhere, for #while loops, at the #end, the parser must
> >>re-read the source to find the #while, like old BASIC dialects did.
> >
> >Not true, fortunately.  It stores the offset within the file where the
> >#while appears and jumps directly to it upon hitting the #end.
> 
> Fortunately, yes.
> 
> >>Perhaps there would be demand for a small program, which unravels #while
> >>loops and macros and creates (huge) include files out of them, taking a
> bit
> >>out of the parsing POV-Ray does.
> >
> >Interesting idea, but I'm not sure you'll save much time, as that's
> >practically the same as what POV already does.  By seeking backward
> >in the file, it essentially duplicates the code as many times as
> >needed.  The only thing you'd save is a few seeks (nearly instantaneous
> >anyway except when it's a macro defined in another file), a few
> >expression evaluations (these could potentially be slow, but usually
> >aren't), and a few symbol lookups on your index variable (very fast.)
> 
> Still, the parser is an interpreter. Even if the allocation of objects takes
> much time, the interpreter also uses time, which Spider's example with the
> pre-computed trees proves. A pre-compiler (generating some kind of p-code,
> optimized for the language, perhaps) would be a nice feature IMHO. A good
> one could even look ahead and optimize (optimise) the allocations (yes I'm
> dreaming again).
now it isn't too much of a dream, searching for exact textures and
materials, grouping in a union and adding once is a pretty big memory
saving....
 
> I myself am used to a very fast compiler (Delphi)
yup, it's nice... Feels very good to have it.

>, so waiting for a parse is
> really annoying (first time I made a longer parse, I thought the program was
> hanging). You (MSVC) C++ guys are propably used to longer compile times <g>.
Hmm, not very much longer, some of the c compilers are astoundishing
fast.
 
> But I agree with the POV-team, that normally the render time is much more of
> a problem, so they should propably optimize (optimise) that first.
Yes, it is the most, but why keep a time hog(if small) when one can get
rid of it ?
yea yea, DIY I hear from someone... Well, I can't, that's why I'm here
and not doing it...



//Spider

Post a reply to this message

Ron Parker wrote:
> 
> On Wed, 3 Feb 1999 21:39:08 +0100, Rudy Velthuis <rve### [at] gmxnet> wrote:
> >But you'll propably have to write a rather extensive parser for this. Once
> >you've done that, the rest won't be hard to implement anymore.
> 
> But if you use POV's built-in parser, and get the code added to the
> official release somehow...  This is how POB format works, but it's
> not official.  All of the issues with binary files go away if you
> use a text file in a known format instead.  The result is something
> that's a little slower parsing than a binary file but not as slow
> as a procedural scene description, more cross-platform than a binary
> file, and backward compatible with existing versions of POV.  It's
> like the C preprocessor: it outputs C code, but it's not C code mere
> mortals would want to read.  In fact, many compilers even have an
> option for "preprocess only."  This is the sort of option I'm
> proposing.
I agree with this point Ron, why do binary? 
A general ansi/ascii format would be far better, and the parse time for
those, around 15 sec for one of my bigger trees of 40 Mb is not so very
much... Not compared to the parse time(I don't want to think about it)

//Spider

Post a reply to this message

Hey, nice thread guys!

It turns out that I've been doing some fiddling
with the parser as well.  It would be good
for someone to rip out the parser, convert
it to C++, make it modular, and support:

1.  Parse Without Render.  (Syntax Check)
2.  Unroll includes, macros, defines and loops to output file.

Probably this scope is acceptable to
the PovTeam, while a binary file format
is not.

Post a reply to this message

Spider schrieb in Nachricht <36B8D2D3.DE77CCE7@bahnhof.se>...
>Lets start commenting on the discussion.
>
>
>> Only difference is the fact, that WinTree does the recursion once (in
Visual
>> Basic),
>*GASP*
>Whatta heck are you accusing me of?
>It is Delphi, admittedly using a very sloppy design, and rxlib, and
>dElphi 3, but... "VISUAL BASIC" ????! *gasp* *choke*

Sorry! No offense intended! I really thought so, but I must have had another
program in mind. I thought I remembered your program needed the VB runtimes.
Must have been one of the other utilities I recently downloaded. And now you
mention it, I remember the RXLib type of spin buttons (the diagonal ones).

To the innocent bystander I *must* explain, that telling someone who is a
Delphi freak, that he is using VB is one of the worst accustions a person
can make <g> (like telling Kenneth Starr voted for Bill Clinton last
election).


>> producing an include file with all objects already spelled out, and
>> the recursive parser does this while parsing. So parsing is definitely
the
>> culprit here, not allocation.

>Yes, it is. That's why I made the prog. the allocation takes around 4-8
>seconds, depending on the size of the tree.

Hmmm. I must definitely have a look at the parser and allocation code one
day. It's said to be plain ANSI C, so I think I can read it well enough.

What would interest me, if there are big differences in parsing/rendering
ratio on various systems, and using different compilers on the same system.
E.g. are the different patches for Win32 almost equally efficient, or is one
more efficient in parsing and the other more in rendering? I think Ron
should be able to answer part of that. I think he uses VC6, while the
official build is done in Watcom C. Perhaps someone should try BCB and BC
too. (Sorry guys, don't know the other platforms well enough).

>> Perhaps there would be demand for a small program, which unravels #while
>> loops and macros and creates (huge) include files out of them, taking a
bit
>> out of the parsing POV-Ray does. This could work like WinTree, but
perhaps a
>> bit more generalized.

[...]

>yes, this might be helpful, but perhaps not good enough. This is a thing
>too look deper into.

I think Ron Parker had some very good suggestions on this.

--
Rudy

Post a reply to this message

Rudy typed : 
> Sorry! No offense intended! I really thought so, but I must have had another
> program in mind. I thought I remembered your program needed the VB runtimes.
> Must have been one of the other utilities I recently downloaded. And now you
> mention it, I remember the RXLib type of spin buttons (the diagonal ones).
hehe, glad you noticed, they are far easier to hit with the cursor, as
well as taht they handle percision a bit better.

> To the innocent bystander I *must* explain, that telling someone who is a
> Delphi freak, that he is using VB is one of the worst accustions a person
> can make <g> (like telling Kenneth Starr voted for Bill Clinton last
> election).
Actully, I tried VB 3 on my old 3.1 system, before delphi time, and
found taht it really sucked... (Have the install disks here somewhere,
no deletium container yet)
 
> >Yes, it is. That's why I made the prog. the allocation takes around 4-8
> >seconds, depending on the size of the tree.
> 
> Hmmm. I must definitely have a look at the parser and allocation code one
> day. It's said to be plain ANSI C, so I think I can read it well enough.
hehe, as I said, I don't have a compiler... :-)
 
> What would interest me, if there are big differences in parsing/rendering
> ratio on various systems, and using different compilers on the same system.
> E.g. are the different patches for Win32 almost equally efficient, or is one
> more efficient in parsing and the other more in rendering? I think Ron
> should be able to answer part of that. I think he uses VC6, while the
> official build is done in Watcom C. Perhaps someone should try BCB and BC
> too. (Sorry guys, don't know the other platforms well enough).
I doubt it is much difference, since a optimization in the compiling
wouldn't be noticeable that very much in the parser... But, I can't
answer this either.. (Anyone here can ?)
 
> >> Perhaps there would be demand for a small program, which unravels #while
> >> loops and macros and creates (huge) include files out of them, taking a
> bit
> >> out of the parsing POV-Ray does. This could work like WinTree, but
> perhaps a
> >> bit more generalized.
> 
> [...]
> 
> >yes, this might be helpful, but perhaps not good enough. This is a thing
> >too look deper into.
> 
> I think Ron Parker had some very good suggestions on this.
Perhaps..
I came to think of it like this :

on every #macro it finds, it places the macro in macroname.par file, and
the parsed call is #included at the parse storage, and the next time, it
is compared, byte/byte to the macro with the same name8and the call) and
if there is a change, it reparses, if there isn't one, it simply
includes the old parsed info.

just an idea on the solving..

//Spider

Post a reply to this message

Some IMP discussion towards fast "Syntax Check"...

> Problem 1 - Determine the minimal fileset required
> for a particular povray command line, or INI file.

 Some initial hacking indicates it's reasonably
 easy to put POV into 'makedepend' mode.  I also
 found that I could bypass the code that reads
 in image files, which is a common parsing
 bottleneck.  Another bottleneck I'd like to
 look at is mesh.

 Macros are the other problem area I can think
 of.  Any others?

Test Script A
                         Files in RAM
Parse          15 sec      11 sec
DependOnly      1 sec       1 sec

Post a reply to this message

Nigel Stewart wrote:
> 
> Some IMP discussion towards fast "Syntax Check"...
> 
> > Problem 1 - Determine the minimal fileset required
> > for a particular povray command line, or INI file.
> 
>  Some initial hacking indicates it's reasonably
>  easy to put POV into 'makedepend' mode.  I also
>  found that I could bypass the code that reads
>  in image files, which is a common parsing
>  bottleneck.  Another bottleneck I'd like to
>  look at is mesh.
> 
>  Macros are the other problem area I can think
>  of.  Any others?
> 
> Test Script A
>                          Files in RAM
> Parse          15 sec      11 sec
> DependOnly      1 sec       1 sec
There is one in large unions, a union of several textured objects put
together... The Fir tree is the one that comes to my mind.

//Spider

Post a reply to this message

Spider wrote:
> 
> Hello, just me and my wild ideas here.
> I have been working a lot with macro's and creating lot's of objects(See
> my fireworks posts)
> While doing this, I have to check lightsourses and fog settings, and
> since the fireworks are lightsources I need to wait for parse every
> time.
> Now I wondered, is there a possibility to add something of a expander to
> pov?
> 
> By storing the parsed data in a new file, ex. example.pov ->example.par
> ->example.tga
> 
> This could be specified with a command line argument, or a setting in
> global_settings {
>   write_parse true
>   load_parse false
> }
> 
> This is only an idea, I'm not sure if it'd work, or if it is implemented
> somewhere already, but I think it might come in hadny in several cases.
> 
> //Spider

Hi,
I don't exactly know where to put my reply as it comments on some parts
of the thread. So it just goes here.

Reading the thread it comes to my mind, that the major problem is
creating a set of similar objects.

I am mostly doing Matlab-programming, which is in a way quite similar to
PovRay. It is a interpreter. Well, that quite all, but in fact, Matlab
is darn slow on loops. Loops are hell, to be honest, and they usually
even kill the memory management. Being specialised on chunks of data,
Matlab is vecotrizing a lot of operations. This could possibly be done
in Povray just as well. So instead of doing some
#while
  sphere{}
#end

thing one could just define an array
#define hallo= array[3][3]{{1,0,0},{2,0,0},{3,0,0}}
and generate three spheres by
sphere{hallo,1}

It's a bit tricky and you possibly have to restrict this feature on some
specific variant of the syntax, but it generally speeds up quite a lot.

Axel

Post a reply to this message