POV-Ray : Newsgroups : povray.programming : POV-Ray parser in Java Server Time
9 Oct 2026 18:46:23 EDT (-0400)
  POV-Ray parser in Java (Message 1 to 28 of 28)  
From: Andrew Wilcox
Subject: POV-Ray parser in Java
Date: 4 Dec 2002 16:06:52
Message: <3dee6e6c$1@news.povray.org>
Would anyone else find this useful?  I started writing it, and I'm past the
hardest part, and am left with just the tedious task of finding all the
keywords defined in POV, and implementing the symantic control of which
keywords are allowed in which objects.

For the most part what I need works, but can anyone give me a good reason to
go ahead and complete this 100%?

Andrew Wilcox


Post a reply to this message

From: Thorsten Froehlich
Subject: Re: POV-Ray parser in Java
Date: 4 Dec 2002 16:25:37
Message: <3dee72d1$1@news.povray.org>
In article <3dee6e6c$1@news.povray.org> , "Andrew Wilcox" 
<awi### [at] unpuzzledcom> wrote:

> For the most part what I need works

Macros?

    Thorsten


Post a reply to this message

From: Andrew Wilcox
Subject: Re: POV-Ray parser in Java
Date: 4 Dec 2002 16:33:00
Message: <3dee748c$1@news.povray.org>
Actually, just parsing expressions is what I consider the hardest part.
Right now, it lexes, tokenizes, and starts objectizing the stream.  It
doesn't actually evaluate directives yet, but this is really just symbol
table manipulation which for the most part isn't too hard.

> > For the most part what I need works
>
> Macros?
>
>     Thorsten


Post a reply to this message

From: Christoph Hormann
Subject: Re: POV-Ray parser in Java
Date: 4 Dec 2002 16:35:09
Message: <3DEE750B.3B747C9B@gmx.de>
Thorsten Froehlich wrote:
> 
> > For the most part what I need works
> 
> Macros?
> 

Functions? ;-)

For a POV-Ray 3.5 compatible parser you really would need to duplicate
half of POV-Ray to successfully parse a POV-Ray scene, even if you leave
out textures.

Christoph

-- 
POV-Ray tutorials, include files, Sim-POV,
HCR-Edit and more: http://www.tu-bs.de/~y0013390/
Last updated 15 Nov. 2002 _____./\/^>_*_<^\/\.______


Post a reply to this message

From: Andrew Wilcox
Subject: Re: POV-Ray parser in Java
Date: 4 Dec 2002 16:42:57
Message: <3dee76e1@news.povray.org>
Maybe I should reclarify.  I don't think actually fully parsing the scene is
feasible anytime soon, especially with functions, but building a parse tree
is.  Which would allow for manipulation of the scene language itself.  This
could then be rewritten and let POV actually do the full parse to generate
the scene.  I can't think of a good reason for actually evaluating the
scene.

AW

"Christoph Hormann" <chr### [at] gmxde> wrote in message
news:3DEE750B.3B747C9B@gmx.de...
>
>
> Thorsten Froehlich wrote:
> >
> > > For the most part what I need works
> >
> > Macros?
> >
>
> Functions? ;-)
>
> For a POV-Ray 3.5 compatible parser you really would need to duplicate
> half of POV-Ray to successfully parse a POV-Ray scene, even if you leave
> out textures.
>
> Christoph
>
> --
> POV-Ray tutorials, include files, Sim-POV,
> HCR-Edit and more: http://www.tu-bs.de/~y0013390/
> Last updated 15 Nov. 2002 _____./\/^>_*_<^\/\.______


Post a reply to this message

From: Vadim Sytnikov
Subject: Re: POV-Ray parser in Java
Date: 7 Jan 2003 11:56:24
Message: <3e1b06b8$1@news.povray.org>
"Andrew Wilcox" <awi### [at] unpuzzledcom> wrote
> Would anyone else find this useful?

Most definitely, no -- if by "useful" you mean something that could be used
for any real-world applications and not just in a students' toy project,
thesis, and the likes.

What (I think) would be indeed highly useful are lex/flex and bison/yacc
conunterparts with licenses more open than GPL. In fact, a good FSM-based
parser is what POV-Ray is currently missing badly, IMHO.


Post a reply to this message

From: Apache
Subject: Re: POV-Ray parser in Java
Date: 7 Jan 2003 13:15:13
Message: <3e1b1931$1@news.povray.org>
Could be very interesting for a remote-POV-Ray computing website. You have a
heavy box running POV-Ray on the webserver with https (ssl) and you can
render stuff from anywhere you are, just using a web-browser with Java
enabled or a java application with sockets. The rendered output comes back
to where you are or is being mailed to an e-mail address. Maybe some thing
like global settings {mailto bil### [at] microsoftcom} would be funny
enhancement for the remote povray thing ?  :-)


Post a reply to this message

From: Fidel viegas
Subject: Re: POV-Ray parser in Java
Date: 7 Jan 2003 18:06:57
Message: <BA410C8C.8D7C%fidel.viegas@artrecognition.co.uk>
in article 3dee76e1@news.povray.org, Andrew Wilcox at awi### [at] unpuzzledcom
wrote on 4/12/02 9:43 pm:

> Maybe I should reclarify.  I don't think actually fully parsing the scene is
> feasible anytime soon, especially with functions, but building a parse tree
> is.  Which would allow for manipulation of the scene language itself.  This
> could then be rewritten and let POV actually do the full parse to generate
> the scene.  I can't think of a good reason for actually evaluating the
> scene.
> 
> AW

Hi Andrew, are you writing the parser by hand? If yes, then you should have
a look at either JFlex and CUP (java equivalent to flex and bison), SableCC
(which generates the parse tree for you) or ANTLR (which allow you to define
tree walkers as well as parsers and scanners). SableCC you can download it
from www.sablecc.org. ANTLR you can download it from www.antlr.org, JFlex
you can download it from www.jflex.de and CUP you can download it from
http://www.cs.princeton.edu/~appel/modern/java/CUP/.

Try SableCC as it is very easy to use.

All the best

Fidel.


Post a reply to this message

From: Andrew Wilcox
Subject: Re: POV-Ray parser in Java
Date: 8 Jan 2003 10:55:30
Message: <3e1c49f2$1@news.povray.org>
Well, I've thought of a couple of potential uses.  My overall goal is trying
to build a render farm program, and there are some tricky points.  One of
the things I am trying to do is determine all the resource files needed by a
POV-Ray scene file.  Things like #include files, and image files for
mappings and such.  To some degree even resources where the name is
generated at parse time could be figured out, unless some super complicated
method is used (like using a function or something).  Also, if done right
you could add syntax which could be parsed and rewritten in standard POV
SDL.  For example adding a #for statement, that gets rewritten as a #while.
It wouldn't be something most people would want to do, but it might make my
life easier sometimes. :-)

AW


Post a reply to this message

From: Thorsten Froehlich
Subject: Re: POV-Ray parser in Java
Date: 8 Jan 2003 16:45:38
Message: <3e1c9c02@news.povray.org>
In article <3e1b06b8$1@news.povray.org> , "Vadim Sytnikov" <syt### [at] rucom>
wrote:

> What (I think) would be indeed highly useful are lex/flex and bison/yacc
> conunterparts with licenses more open than GPL. In fact, a good FSM-based
> parser is what POV-Ray is currently missing badly, IMHO.

No, that isn't correct.  You should not make such bold claims without
profiling the code first!  And this issue has been discussed often in the
past.

    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org


Post a reply to this message

From: Thorsten Froehlich
Subject: Re: POV-Ray parser in Java
Date: 8 Jan 2003 17:05:37
Message: <3e1ca0b1$1@news.povray.org>
In article <BA410C8C.8D7C%fid### [at] artrecognitioncouk> , Fidel viegas
<fid### [at] artrecognitioncouk>  wrote:

> http://www.cs.princeton.edu/~appel/modern/java/CUP/.

<http://www.cs.princeton.edu/~appel/modern/>

Given that this link goes to a subpage of one of the worst introductory
compiler construction books ever written*, content of that site cannot
really be considered credible or reliable!

    Thorsten



* It is written using writing techniques of those "Learn X in 21 days
books".  All three editions are copy and paste of the other editions, with
all errors.  Completely ignoring language specialties on the implementation
side, with neither head or tail to either the C or Java books (haven't read
the ML edition yet, and don't plan to do it).  Examples contain more errors
that correct lines (referring to the 1st C edition).  And the book does a
great job obscuring the important stuff while setting focus on irrelevant
aspects in the theory sections.  In short 544 pages of overpriced paper with
ink sprinkled on it.

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org


Post a reply to this message

From: Christopher James Huff
Subject: Re: POV-Ray parser in Java
Date: 8 Jan 2003 17:34:52
Message: <cjameshuff-59D59A.17250408012003@netplex.aussie.org>
In article <3e1c9c02@news.povray.org>,
 "Thorsten Froehlich" <tho### [at] trfde> wrote:

> No, that isn't correct.  You should not make such bold claims without
> profiling the code first!  And this issue has been discussed often in the
> past.

Profiling?
I thought he was talking about the ability to create a grammar for other 
people to use without having to use POV source code, and ease of parser 
creation/maintenance. Does profiling have some use other than 
performance tuning?

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: chr### [at] tagpovrayorg
http://tag.povray.org/


Post a reply to this message

From: Thorsten
Subject: Re: POV-Ray parser in Java
Date: 8 Jan 2003 17:55:05
Message: <web.3e1cabf6e8c03b1d37401e050@news.povray.org>
Christopher James Huff wrote:
>In article <3e1c9c02[at]news.povray.org>,
> "Thorsten Froehlich" <tho### [at] trfde> wrote:
>
>> No, that isn't correct.  You should not make such bold claims without
>> profiling the code first!  And this issue has been discussed often in the
>> past.
>
>Profiling?
>I thought he was talking about the ability to create a grammar for other
>people to use without having to use POV source code, and ease of

No, he implied that POV-Ray is missing a fast parser in the part I quoted.

Regarding a grammar: Grammars in a format assuming a specific parser
generator are hardly very useful. And formal grammars cannot be directly
read by any parser generator. So no need for yet another grammar format,
POV-Ray already uses its own (informal one) in the documentation.

    Thorsten


Post a reply to this message

From: Andrew Wilcox
Subject: Re: POV-Ray parser in Java
Date: 8 Jan 2003 18:39:19
Message: <3e1cb6a7@news.povray.org>
I do have to say, that the documentation does a great job of defining very
clearly the exact syntax POV uses.  It really does.

AW


Post a reply to this message

From: fidel viegas
Subject: Re: POV-Ray parser in Java
Date: 8 Jan 2003 19:38:01
Message: <3E1CC2E4.2090203@nodomain.com>
Thorsten Froehlich wrote:
> In article <BA410C8C.8D7C%fid### [at] artrecognitioncouk> , Fidel viegas
> <fid### [at] artrecognitioncouk>  wrote:
> 
> 
>>http://www.cs.princeton.edu/~appel/modern/java/CUP/.
> 
> 
> <http://www.cs.princeton.edu/~appel/modern/>
> 
> Given that this link goes to a subpage of one of the worst introductory
> compiler construction books ever written*, content of that site cannot
> really be considered credible or reliable!

I do agree that the book isn't that good, but the link wasn't meant for 
the book, but for the parser generator. I personally, do not use it, I 
just gave a couple of links of parser generators I know. I use ANTRL 
(which generates code for C++, C# and other languages as well as Java) 
and  SableCC.

All the best

Fidel.

PS: The bible for compiler writing is the dragon book (Compilers: 
Principles, Techniques and Tools, by Alfred Aho), no doubt about it.


Post a reply to this message

From: Mark Wagner
Subject: Re: POV-Ray parser in Java
Date: 9 Jan 2003 00:12:42
Message: <pan.2003.01.09.05.11.26.238099.265@gte.net>
On Wed, 08 Jan 2003 10:54:48 -0500, Andrew Wilcox quoth:

> Well, I've thought of a couple of potential uses.  My overall goal is
> trying to build a render farm program, and there are some tricky points.
>  One of the things I am trying to do is determine all the resource files
> needed by a POV-Ray scene file.  Things like #include files, and image
> files for mappings and such.  To some degree even resources where the
> name is generated at parse time could be figured out, unless some super
> complicated method is used (like using a function or something).

The Parse_String() macro will probably trip any such program up.  It not
only generates the names of the files it includes at runtime, it also
generates the files themselves at that time.

-- 
Mark


Post a reply to this message

From: Vadim Sytnikov
Subject: Re: POV-Ray parser in Java
Date: 9 Jan 2003 01:48:16
Message: <3e1d1b30@news.povray.org>
"Thorsten" <nomail@nomail> wrote:
> No, he implied that POV-Ray is missing a fast parser in the part I quoted.

Correct.

> Regarding a grammar: Grammars in a format assuming a specific parser
> generator are hardly very useful. And formal grammars cannot be directly
> read by any parser generator.

Now it is I that have to say that your statement is too bold without trying
to express POV-Ray's grammar in yacc. Did you do that? I have to admit that
I did not do that either, but my great experience in the area of compilers'
construction suggests that POV's *formal* grammar might as well turn out to
be the *working* one. Here, I imply nothing about its performance, just more
or less "direct use". But -- see below.

Yacc's/Bison's grammar is as close to BNF as it could probably be. So, if I
got you right, you implied that before a grammar could be made unambiguous
it would have to be transformed (in an attempt to resolve reduce-reduce
conflicts; shift-reduce are generally OK; here, I imply bottom-up analysis,
of course) to such a degree that it would become virtually unusable for a
human reader (e.g. one of those creating third-party SW to use with POV)?

I disagree. My point is that (i) POV-Ray's SDL structure is very linear, and
(ii) the number of reserved words it currently has suggests that performance
gain from using an FSM-based *scanner* would be even greater than the
languages with far fewer nmbers of keywords enjoy (e.g. C/C++).

The golden mean between your and my approaches would probably to make an
FSM-based POV SDL scanner (which *would* make difefrence, I swear) but leave
parser intact.


Post a reply to this message

From: Thorsten Froehlich
Subject: Re: POV-Ray parser in Java
Date: 9 Jan 2003 04:12:12
Message: <3e1d3cec$1@news.povray.org>
In article <3e1d1b30@news.povray.org> , "Vadim Sytnikov" <syt### [at] rucom>
wrote:

> Now it is I that have to say that your statement is too bold without trying
> to express POV-Ray's grammar in yacc. Did you do that? I have to admit that

I know yacc and I know the POV-Ray grammar.  That is enough for me to not
try it myself.

> I disagree. My point is that (i) POV-Ray's SDL structure is very linear
<snip>
> languages with far fewer nmbers of keywords enjoy (e.g. C/C++).

I agree with you that POV-Ray is different from programming languages in
this regard.

> The golden mean between your and my approaches would probably to make an
> FSM-based POV SDL scanner (which *would* make difefrence, I swear) but leave
> parser intact.

You should really profile the code.  Then you would find that the matching
of keywords isn't the bottleneck at all.  The simple way of reading from
file is the real problem.  Make the fgetc and ungetc ten times faster and
you end up with a much faster parser.

    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org


Post a reply to this message

From: Vadim Sytnikov
Subject: Re: POV-Ray parser in Java
Date: 9 Jan 2003 04:38:34
Message: <3e1d431a@news.povray.org>
"Thorsten Froehlich" <tho### [at] trfde> wrote:
> You should really profile the code.  Then you would find that the matching
> of keywords isn't the bottleneck at all.  The simple way of reading from
> file is the real problem.  Make the fgetc and ungetc ten times faster and
> you end up with a much faster parser.

In this respect, I can suggest an easy, virtually free (in terms of efforts)
solution: (i) use getc() instead of fgetc(), and (ii) always #include
<stdio.h> in the modules that use getc(). In such a case, you end up using
very efficient macro instead of a function; five to ten times or so as fast;
no side effects. This is an ANSI behaviour, not a VC/GCC/etc. hack.

I am still a bit suspicious as to the real bottlenecks though... but you're
right, there is no point in arguing w/o profiling it. One day I will do this
again, for sure... As you did read my povpro info, you should be aware that
I did that (profiling) in the past, but I have to admit that I did not pay
any attantion to parsing step at the time, at all. Well, one day, when I
have enough time on my hands...


Post a reply to this message

From: Thorsten Froehlich
Subject: Re: POV-Ray parser in Java
Date: 9 Jan 2003 06:05:28
Message: <3e1d5778@news.povray.org>
In article <3e1d431a@news.povray.org> , "Vadim Sytnikov" <syt### [at] rucom>
wrote:

> In this respect, I can suggest an easy, virtually free (in terms of efforts)
> solution: (i) use getc() instead of fgetc(), and (ii) always #include
> <stdio.h> in the modules that use getc(). In such a case, you end up using
> very efficient macro instead of a function; five to ten times or so as fast;
> no side effects. This is an ANSI behaviour, not a VC/GCC/etc. hack.

This is only true for ISO C, not ISO C++.  In C++ all C macros are required
to be functions; and when using "cname" instead of "name.h" headers they are
even placed in the std namespace.  POV-Ray 3.5 is a C++ program.

Apart from that the lastest revision of C, ISO C 99 explicitly notes that
getc may be either a function or a macro.  In reality, it is even so that
some libraries implement it in terms of exactly the same code fgetc uses...

> I am still a bit suspicious as to the real bottlenecks though... but you're
> right, there is no point in arguing w/o profiling it. One day I will do this
> again, for sure... As you did read my povpro info, you should be aware that
> I did that (profiling) in the past, but I have to admit that I did not pay
> any attantion to parsing step at the time, at all. Well, one day, when I
> have enough time on my hands...

Takes a scene that takes relatively long to parse like isocacti.pov and
render only a very small image.  You still get some trace and isosurface
stuff in there, but still the result will be clear.

    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org


Post a reply to this message

From: Vadim Sytnikov
Subject: Re: POV-Ray parser in Java
Date: 9 Jan 2003 06:34:08
Message: <3e1d5e30@news.povray.org>
"Thorsten Froehlich" <tho### [at] trfde> wrote:
> This is only true for ISO C, not ISO C++.  In C++ all C macros are
required
> to be functions; and when using "cname" instead of "name.h" headers they
are
> even placed in the std namespace.  POV-Ray 3.5 is a C++ program.

Strictly speaking, that's so, but POV-Ray still uses <stdio.h>, and I see no
reasons not to just change fgetc() to getc() -- if that is indeed a
bottleneck, as you said.

> Apart from that the lastest revision of C, ISO C 99 explicitly notes that
> getc may be either a function or a macro.  In reality, it is even so that
> some libraries implement it in terms of exactly the same code fgetc
uses...

IIRC, it (ISO C) did so for ages. And yet in reality all the compilers I'm
aware of always had a macro version of getc(). My current versions of VC and
GCC certainly do. ICC (Intel's Proton) that I used to run just used MS's
include files, so it also did. So if fgetc() is a bottleneck (as you said),
and you just change fgetc() to getc(), we would have free speed improvement.
It won't help my own version though, since I use multi-threading, and
getc/putc macros are, obviously, unusable in that mode. But as to the
official version... Or are use using MT library as well?

BTW, there are currently only 4 calls of fgetc() in POV-Ray sources (inline
function in file_pov.h, plus method of pov_istream_class in file_pov.cpp).


Post a reply to this message

From: Thorsten Froehlich
Subject: Re: POV-Ray parser in Java
Date: 9 Jan 2003 07:23:41
Message: <3e1d69cd@news.povray.org>
In article <3e1d5e30@news.povray.org> , "Vadim Sytnikov" <syt### [at] rucom>
wrote:

> IIRC, it (ISO C) did so for ages. And yet in reality all the compilers I'm
> aware of always had a macro version of getc(). My current versions of VC and

In C mode they indeed have.  But having the macro usually causes problems in
C++ compilers, so watch carefully if the macro isn't an inline function when
compiling as C++ code.

> GCC certainly do. ICC (Intel's Proton) that I used to run just used MS's
> include files, so it also did. So if fgetc() is a bottleneck (as you said),
> and you just change fgetc() to getc(), we would have free speed improvement.

The Intel compiler usually uses the header files of its host, doesn't it?

> It won't help my own version though, since I use multi-threading, and
> getc/putc macros are, obviously, unusable in that mode. But as to the
> official version... Or are use using MT library as well?

I don't know if the current Windows version needs to or not, the Mac version
certainly does need multithreaded library versions.

> BTW, there are currently only 4 calls of fgetc() in POV-Ray sources (inline
> function in file_pov.h, plus method of pov_istream_class in file_pov.cpp).

Yes.

    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org


Post a reply to this message

From: Thorsten Froehlich
Subject: Re: POV-Ray parser in Java
Date: 14 Jan 2003 07:18:55
Message: <3e24002f@news.povray.org>
In article <3e1d1b30@news.povray.org> , "Vadim Sytnikov" <syt### [at] rucom>
wrote:

> I disagree. My point is that (i) POV-Ray's SDL structure is very linear, and
> (ii) the number of reserved words it currently has suggests that performance
> gain from using an FSM-based *scanner* would be even greater than the
> languages with far fewer nmbers of keywords enjoy (e.g. C/C++).

As we agreed on the scanner part already, here is another good point against
Yacc.  Obviously its grammars are hard to maintain and there is hardly a
worthwhile speed gain, otherwise the December 27th announcement on
<http://gcc.gnu.org/> would not make sense!

    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org


Post a reply to this message

From: Vadim Sytnikov
Subject: Re: POV-Ray parser in Java
Date: 14 Jan 2003 09:42:35
Message: <3e2421db$1@news.povray.org>
"Thorsten Froehlich" <tho### [at] trfde> wrote:
>
> As we agreed on the scanner part already, here is another good point
against
> Yacc.  Obviously its grammars are hard to maintain and there is hardly a
> worthwhile speed gain, otherwise the December 27th announcement on
> <http://gcc.gnu.org/> would not make sense!

Well, interesting announcement, indeed.

The grammars are as hard to maintain as formal language definitions are. I
mean, the accurate description of language *semantics*, not just syntax.
This is a very complicated matter... I, personally, started my compiler
construction education with "The Disign and Construction of Compilers"
(Hunter, 1981), which was published just 4 years before the "Compilers:
Principles, Techniques, and Tools" (Aho, Sethi, Ullman; 1985). Just 4-year
span, and yet there were a fundamental difference: the first book's author
had no idea as to how to describe language semantics (vs syntax), at all.
There were some vague and virtually useless proposition of meta-grammars,
and that's all. While the second book introduced annotated grammars
(grammars with attributes). Here, I refer to the "Red Dragon" edition; I
have no idea whether earlier editions had that as well. What I am trying to
say is that, to the best of my knowledge, attributed grammars are the only
way (discovered to date) to accurately represent language *semantics*.

The annotated (attributed) grammars are essentially what Yacc/Bison and
similar systems implement. The "attributes" (from the compiler construction
theory) are those "rules" enclosed in curly braces that you put into the
Yacc rules. So, as to maintaining language *definition*, Yacc rules are
unbeatable -- respective implementations are as close to theory as possible.
Consequently, there is a possibility to automatically generate highly
efficient language parsers, far more efficient than a casual compiler writer
may produce. Or course, that may not be the case with those Los Alamos guys.
In this respects, to maintain POV-Ray grammar as a Yacc one still seems to
be highly desirable -- IMHO.

One thing that used to always bother me a lot was the extreme difficulty of
maintaining the "context" in bottom-up analysis, i.e. in Yacc/Bison
grammars... But as soon as I got accustomed to the use of fake non-terminals
to track context, life became significantly easier. To me, that is still the
only noticable weakness of real-life LR1/LALR1 grammars used in bottom-up
analysis. I bet those guys used *much* more hacks to make a working C++
LL1-based parser... Once you start to write recursive-descend parser for a
real-world application, you quickly notice that you are forced to patch it
more and more, pass more and more information to and from those recursive
calls (I suspect that that is what "hand-crafted" stands for in that
announcement :-) -- in strict accordance with the theory, that states that
LL1 grammars are suitable for a much smaller set of languages than LALR1
(let alone LR1). Fortunately, as I noted earlier, this does not have such a
strong impact on POV-Ray -- due to its more or less linear nature. BTW, how
do you think: to what extent the linear POV-Ray SDL structure is due to its
LL1-constrained parser? :-)

Well, back to practice... I sincerely think that a good profiling of
POV-Ray's *scanner* has to be done; I promise to do that (again) myself, one
day, as I have time. To date, speaking of parse time, the only thing that
*really* got in my way was parsing of really big meshes -- which obviously
cannot be due to parser's design, but just because of its implementation.
Maybe you're right with your getc()/ungetc() observation. If that is indeed
so, the radical solution might as well include the implementation of 3DS
mesh readers (why not? we do use binary images, and there seems to be no
fundamental differences). Other (non-mesh) things just parse much longer
that I would have liked; in acceptable time, but much longer than they ought
to. *This* might as well be due to scanner design; so some profiling is due,
definitely.


Post a reply to this message

From: Mike Lundy
Subject: Re: POV-Ray parser in Java
Date: 13 May 2003 18:33:42
Message: <3EC17301.40002@netscape.net>
/************************************************************************/
/*                             Code section                             */
/************************************************************************/

%{

/* Syntax highlite a POV-Ray 3.5 script using FLEX. Output is HTML.
  * by Mike Lundy - May 10, 2003
  *
  * To complie:
  *  flex pov.yy
  *  gcc lex.yy.c -lfl -opov2html
  *
  * Usage: pov2html <filename.pov >filename.html
  *    or: pov2html filename.pov >filename.html
  *    or: pov2html filename.pov filename.html
  */

#define PAGE_COLOR              "#FDF5E6"
#define TEXT_COLOR              "#000000"
#define KEYWORD_COLOR           "#0000BF"
#define DIRECTIVE_COLOR         "#0000FF"
#define COMMENT_COLOR           "#007F00"
#define STRING_COLOR            "#BF0000"
#define INTEGER_COLOR           "#800080"
#define FLOAT_COLOR             "#800080"
#define OPERATOR_COLOR          "#FF0000"
#define DELIMITER_COLOR         "#008080"

#define FONT_TAG(tExT, cOlOr)   fprintf(yyout, "<FONT COLOR=\"%s\">%s</FONT>",
(cOlOr), (tExT));
#define ECHO                    fprintf(yyout, "%s", yytext);
#define AUTHOR                  "mwl### [at] netscapenet"

/* tidy HTML source on '//' type comments */
char *kill_EOL (char* line)
{
     char *p1 = line;
     int len = 0;
     while(*p1++) len++;
     if( (line[--len]) == '\n') { line[len] = 0; }
     return line;
}

%}


/************************************************************************/
/*                          Definition section                          */
/************************************************************************/

%option noyywrap

DIGIT   [0-9]

INTEGER {DIGIT}+

FLOAT   ({DIGIT}+"."{DIGIT}*)|("."{DIGIT}+)|({DIGIT}+".")

STRING \"[^"]*[\"]*\"

OPERATOR [+*/|!:?=]|"-"

DELIMITER [;{}]

DIRECTIVE
break|case|debug|declare|default|else|end|error|fclose|fopen|if|ifdef|ifndef|include|local|macro|range|read|render|statistics|switch|undef|version|warning|while|write

%x comment1
%x comment2


/************************************************************************/
/*                             Rules section                            */
/************************************************************************/

%%

"/*"                     { BEGIN(comment1); fprintf(yyout, "<FONT COLOR=\"%s\">%s",
COMMENT_COLOR, yytext); }
<comment1>[^*\n]*        ECHO;
<comment1>[^*\n]*\n      ECHO;
<comment1>"*"+[^*/\n]*   ECHO;
<comment1>"*"+[^*/\n]*\n ECHO;
<comment1>"*"+"/"        { fprintf(yyout, "%s%s", yytext, "</FONT>"); BEGIN(INITIAL);
}

"//"                     { BEGIN(comment2); fprintf(yyout, "<FONT COLOR=\"%s\">%s",
COMMENT_COLOR, yytext);; }
<comment2>\n+            { fprintf(yyout, "%s%s", kill_EOL(yytext), "</FONT>\n");
BEGIN(INITIAL); }

<comment1,comment2><     { fprintf(yyout, "%s", "<"  ); }
<comment1,comment2>>     { fprintf(yyout, "%s", ">"  ); }
<comment1,comment2>&     { fprintf(yyout, "%s", "&" ); }
<comment1,comment2>\"    { fprintf(yyout, "%s", """); }

<INITIAL><               { FONT_TAG("<",   OPERATOR_COLOR); }
<INITIAL>>               { FONT_TAG(">",   OPERATOR_COLOR); }
<INITIAL>&               { FONT_TAG("&",  OPERATOR_COLOR); }
<INITIAL>\"              { FONT_TAG(""", STRING_COLOR);   }

{STRING}                 { FONT_TAG(yytext, STRING_COLOR);    }
{OPERATOR}               { FONT_TAG(yytext, OPERATOR_COLOR);  }
{INTEGER}                { FONT_TAG(yytext, INTEGER_COLOR);   }
{FLOAT}                  { FONT_TAG(yytext, FLOAT_COLOR);     }
{DELIMITER}              { FONT_TAG(yytext, DELIMITER_COLOR); }
#+[ \t]*{DIRECTIVE}+     { FONT_TAG(yytext, DIRECTIVE_COLOR); }

aa_level                        |
aa_threshold                    |
abs                             |
absorption                      |
accuracy                        |
acos                            |
acosh                           |
adaptive                        |
adc_bailout                     |
agate                           |
agate_turb                      |
all                             |
all_intersections               |
alpha                           |
altitude                        |
always_sample                   |
ambient                         |
ambient_light                   |
angle                           |
aperture                        |
append                          |
arc_angle                       |
area_light                      |
array                           |
asc                             |
ascii                           |
asin                            |
asinh                           |
assumed_gamma                   |
atan                            |
atan2                           |
atanh                           |
autostop                        |
average                         |
b_spline                        |
background                      |
bezier_spline                   |
bicubic_patch                   |
black_hole                      |
blob                            |
blue                            |
blur_samples                    |
bounded_by                      |
box                             |
boxed                           |
bozo                            |
break                           |
brick                           |
brick_size                      |
brightness                      |
brilliance                      |
bump_map                        |
bump_size                       |
bumps                           |
camera                          |
case                            |
caustics                        |
ceil                            |
cells                           |
charset                         |
checker                         |
chr                             |
circular                        |
clipped_by                      |
clock                           |
clock_delta                     |
clock_on                        |
collect                         |
color                           |
color_map                       |
colour                          |
colour_map                      |
component                       |
composite                       |
concat                          |
cone                            |
confidence                      |
conic_sweep                     |
conserve_energy                 |
contained_by                    |
control0                        |
control1                        |
coords                          |
cos                             |
cosh                            |
count                           |
crackle                         |
crand                           |
cube                            |
cubic                           |
cubic_spline                    |
cubic_wave                      |
cutaway_textures                |
cylinder                        |
cylindrical                     |
debug                           |
declare                         |
default                         |
defined                         |
degrees                         |
density                         |
density_file                    |
density_map                     |
dents                           |
df3                             |
difference                      |
diffuse                         |
dimension_size                  |
dimensions                      |
direction                       |
disc                            |
dispersion                      |
dispersion_samples              |
dist_exp                        |
distance                        |
div                             |
double_illuminate               |
eccentricity                    |
else                            |
emission                        |
end                             |
error                           |
error_bound                     |
evaluate                        |
exp                             |
expand_thresholds               |
exponent                        |
exterior                        |
extinction                      |
face_indices                    |
facets                          |
fade_color                      |
fade_colour                     |
fade_distance                   |
fade_power                      |
falloff                         |
falloff_angle                   |
false                           |
fclose                          |
file_exists                     |
filter                          |
final_clock                     |
final_frame                     |
finish                          |
fisheye                         |
flatness                        |
flip                            |
floor                           |
focal_point                     |
fog                             |
fog_alt                         |
fog_offset                      |
fog_type                        |
fopen                           |
form                            |
frame_number                    |
frequency                       |
fresnel                         |
function                        |
gather                          |
gif                             |
global_lights                   |
global_settings                 |
gradient                        |
granite                         |
gray                            |
gray_threshold                  |
green                           |
height_field                    |
hexagon                         |
hf_gray_16                      |
hierarchy                       |
hypercomplex                    |
hollow                          |
if                              |
ifdef                           |
iff                             |
ifndef                          |
image_height                    |
image_map                       |
image_pattern                   |
image_width                     |
include                         |
initial_clock                   |
initial_frame                   |
inside                          |
inside_vector                   |
int                             |
interior                        |
interior_texture                |
internal                        |
interpolate                     |
intersection                    |
intervals                       |
inverse                         |
ior                             |
irid                            |
irid_wavelength                 |
isosurface                      |
jitter                          |
jpeg                            |
julia                           |
julia_fractal                   |
lambda                          |
lathe                           |
leopard                         |
light_group                     |
light_source                    |
linear_spline                   |
linear_sweep                    |
ln                              |
load_file                       |
local                           |
location                        |
log                             |
look_at                         |
looks_like                      |
low_error_factor                |
macro                           |
magnet                          |
major_radius                    |
mandel                          |
map_type                        |
marble                          |
material                        |
material_map                    |
matrix                          |
max                             |
max_extent                      |
max_gradient                    |
max_intersections               |
max_iteration                   |
max_sample                      |
max_trace                       |
max_trace_level                 |
media                           |
media_attenuation               |
media_interaction               |
merge                           |
mesh                            |
mesh2                           |
metallic                        |
method                          |
metric                          |
min                             |
min_extent                      |
minimum_reuse                   |
mod                             |
mortar                          |
natural_spline                  |
nearest_count                   |
no                              |
no_bump_scale                   |
no_image                        |
no_reflection                   |
no_shadow                       |
noise_generator                 |
normal                          |
normal_indices                  |
  normal_map                     |
normal_vectors                  |
number_of_waves                 |
object                          |
octaves                         |
off                             |
offset                          |
omega                           |
omnimax                         |
on                              |
once                            |
onion                           |
open                            |
orient                          |
orientation                     |
orthographic                    |
panoramic                       |
parallel                        |
parametric                      |
pass_through                    |
pattern                         |
perspective                     |
pgm                             |
phase                           |
phong                           |
phong_size                      |
photons                         |
pi                              |
pigment                         |
pigment_map                     |
pigment_pattern                 |
planar                          |
plane                           |
png                             |
point_at                        |
poly                            |
poly_wave                       |
polygon                         |
pot                             |
pow                             |
ppm                             |
precision                       |
precompute                      |
pretrace_end                    |
pretrace_start                  |
prism                           |
prod                            |
projected_through               |
pwr                             |
quadratic_spline                |
quadric                         |
quartic                         |
quaternion                      |
quick_color                     |
quick_colour                    |
quilted                         |
radial                          |
radians                         |
radiosity                       |
radius                          |
rainbow                         |
ramp_wave                       |
rand                            |
range                           |
ratio                           |
read                            |
reciprocal                      |
recursion_limit                 |
red                             |
reflection                      |
reflection_exponent             |
refraction                      |
render                          |
repeat                          |
rgb                             |
rgbf                            |
rgbft                           |
rgbt                            |
right                           |
ripples                         |
rotate                          |
roughness                       |
samples                         |
save_file                       |
scale                           |
scallop_wave                    |
scattering                      |
seed                            |
select                          |
shadowless                      |
sin                             |
sine_wave                       |
sinh                            |
size                            |
sky                             |
sky_sphere                      |
slice                           |
slope                           |
slope_map                       |
smooth                          |
smooth_triangle                 |
solid                           |
sor                             |
spacing                         |
specular                        |
sphere                          |
sphere_sweep                    |
spherical                       |
spiral1                         |
spiral2                         |
spline                          |
split_union                     |
spotlight                       |
spotted                         |
sqr                             |
sqrt                            |
statistics                      |
str                             |
strcmp                          |
strength                        |
strlen                          |
strlwr                          |
strupr                          |
sturm                           |
substr                          |
sum                             |
superellipsoid                  |
switch                          |
sys                             |
t                               |
tan                             |
tanh                            |
target                          |
text                            |
texture                         |
texture_list                    |
texture_map                     |
tga                             |
thickness                       |
threshold                       |
tiff                            |
tightness                       |
tile2                           |
tiles                           |
tolerance                       |
toroidal                        |
torus                           |
trace                           |
transform                       |
translate                       |
transmit                        |
triangle                        |
triangle_wave                   |
true                            |
ttf                             |
turb_depth                      |
turbulence                      |
type                            |
u                               |
u_steps                         |
ultra_wide_angle                |
undef                           |
union                           |
up                              |
use_alpha                       |
use_color                       |
use_colour                      |
use_index                       |
utf8                            |
uv_indices                      |
uv_mapping                      |
uv_vectors                      |
v                               |
v_steps                         |
val                             |
variance                        |
vaxis_rotate                    |
vcross                          |
vdot                            |
version                         |
vertex_vectors                  |
vlength                         |
vnormalize                      |
vrotate                         |
vstr                            |
vturbulence                     |
warning                         |
warp                            |
water_level                     |
waves                           |
while                           |
width                           |
wood                            |
wrinkles                        |
write                           |
x                               |
y                               |
yes                             |
z                        { FONT_TAG(yytext, KEYWORD_COLOR); }

[a-zA-Z0-9_#]+         |
<*>.|\n                ECHO;

<<EOF>> yyterminate();
%%


/************************************************************************/
/*                          User code section                           */
/************************************************************************/

#include <time.h>

/* return current system date & time as char ptr */
char* get_time(void)
{
     static char *the_time_str;

     time_t the_time;
     time(&the_time);
     the_time_str = ctime(&the_time);
     return(the_time_str);
}

int main(int argc, char **argv)
{
     if (!yyin) yyin = stdin;
     if (!yyout) yyout = stdout;

     if(argc >= 2) {
         if( (yyin = fopen (argv[1], "r")) == NULL) {
             perror(argv[1]);
             return(1);
         }
         if(argc == 3) {
             if( (yyout = fopen (argv[2], "w")) == NULL) {
                 perror(argv[2]);
                 return(2);
             }
         }
     }

     fprintf(yyout, "<HTML>\n"
                    "<HEAD>\n"
                    "  <META NAME=\"GENERATOR\" CONTENT=\"%s by %s %s\">\n"
                    "  <TITLE>POV-Ray SDL: %s</TITLE>\n"
                    "</HEAD>\n"
                    "<BODY BGCOLOR=\"%s\" TEXT=\"%s\">\n\n"
                    "<PRE>\n",
                    argv[0], AUTHOR, __DATE__, get_time(), PAGE_COLOR, TEXT_COLOR );
     yylex();

     fprintf(yyout, "</PRE>\n"
                    "<BR>\n"
                    "</BODY>\n"
                    "</HTML>\n" );
     return(0);
}












Andrew Wilcox wrote:
> Would anyone else find this useful?  I started writing it, and I'm past the
> hardest part, and am left with just the tedious task of finding all the
> keywords defined in POV, and implementing the symantic control of which
> keywords are allowed in which objects.
> 
> For the most part what I need works, but can anyone give me a good reason to
> go ahead and complete this 100%?
> 
> Andrew Wilcox
> 
>


Post a reply to this message

From: ABX
Subject: Re: POV-Ray parser in Java
Date: 13 May 2003 18:38:06
Message: <hns2cvcig9pvdbd8novhm0fqnnkt1glfgh@4ax.com>
On Tue, 13 May 2003 15:34:41 -0700, Mike Lundy <mwl### [at] netscapenet> wrote:
> /* Syntax highlite a POV-Ray 3.5 script using FLEX. Output is HTML.
>  * by Mike Lundy - May 10, 2003

Your post does not asnwered to "implementing the symantic control of which
keywords are allowed in which objects" as it was asked in original post. Any
reason to such strange wasting of news server space ? Any comment why you second
time posted (still in wrong group IMO) the same code instead of referencing
previous ?

ABX


Post a reply to this message

From: Mike Lundy
Subject: Re: POV-Ray parser in Java
Date: 13 May 2003 18:52:56
Message: <3EC17785.8040300@netscape.net>
Hi,

There's many replies and one cannot review them all completely... In any event, it's
trivial... and I'm sure we'll get along fine in another thread :-)

Cheers,

Mike





> 
> 
> Your post does not asnwered to "implementing the symantic control of which
> keywords are allowed in which objects" as it was asked in original post. Any
> reason to such strange wasting of news server space ? Any comment why you second
> time posted (still in wrong group IMO) the same code instead of referencing
> previous ?
> 
> ABX


Post a reply to this message

From: Christopher James Huff
Subject: Re: POV-Ray parser in Java
Date: 14 May 2003 14:24:26
Message: <cjameshuff-348898.14244714052003@netplex.aussie.org>
In article <3EC### [at] netscapenet>,
 Mike Lundy <mwl### [at] netscapenet> wrote:

> There's many replies and one cannot review them all completely... In any 
> event, it's trivial... and I'm sure we'll get along fine in another thread 
> :-)

It's trivial to follow the posting rules. Posts with attachments of any 
kind should go in a binaries group, a pure code post should not go in a 
discussion group (as mentioned, the utilities groups would have been 
more appropriate for this), and there just isn't any reason to make 
multiple copies of the same post in the same group. On top of this, your 
post doesn't really fit the subject of the thread and you gave 
practically no explanation. Please review the posting guidelines in the 
povray.announce.frequently-asked-questions group, particularly these:

Welcome To The POV-Ray News Groups
Where can I post my binary or text file?
Cross-Posting Guidelines for news.povray.org
(Actually references multiposting, posting the same message in multiple 
groups...nobody anticipated somebody doing it within one group.)

-- 
Christopher James Huff <cja### [at] earthlinknet>
http://home.earthlink.net/~cjameshuff/
POV-Ray TAG: chr### [at] tagpovrayorg
http://tag.povray.org/


Post a reply to this message

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.