I have built a list of POV-Ray keywords, the first step toward a Scintilla 
lexer.
I have done it manually, reading carefully the official Quick Reference and 
typing all keywords I found, trying to classify them by use, removing 
duplicates.
Then I flattened the file (one keyword per line), sorted it and removed 
forgotten duplicates.
Now the fun part: I wrote a small Lua script to parse Chaffe's quick 
reference (excellent work, very nice and clear). My script extracted from 
the HTML file all words in bold (between <b> and </b>).
After cleaning, I compared both files side by side in Excel.

It seems I was careful enough, with only one keyword forgotten.

Now, I may have extracted these keywords from tokenize.cpp, but I started 
this work before having the sources, and my method allowed me to have more 
control on classification.

About my classification:
It is quite arbitrary, but actually we can use several methods anyway.
I started to classify by objects, ITEMS, MODIFIERS, and functions.
But I felt that it wasn't very consistent. Maybe I missed the signifiance of 
these terms.

Anyway, I chose to classify by language directive (#xxx, the lexer will 
detect them), Objects & CSG & Appearance (followed by soft braces), Types & 
Modifiers (no param or boolean param (ie. on/off, yes/no, often optional)), 
Modifiers & Items (with params), Predefined Identifiers, and POV-Ray 
functions.

In the Modifiers & Items section, I added the kind of param, although it 
will be removed from the syntax file.

The idea is to color code the keywords so that when the user will type them, 
it will see at a glance what kind of (non-)parameters he should type.
Note that some keywords have dual use. For example, cylinder is both an 
object (followed by "{ lot of params }") and a type of camera and light (no 
param). In this case, perhaps "cylindrical" would have been better, like the 
"spherical" keyword. But I understand history and compatibility prevail on 
consistency. I saw also bezier_spline and b_spline...

So my color coding based on use of keyword will sometime fail (unless doing 
a full lexer! Forget it!)... Not a big problem.

OK, here is the result of my hard work :-)
I give it in case somebody else would want to make a syntax file for another 
text editor (TextPad, CodeGenie, UltraEdit, whatever).
I have not yet sorted the keywords, they are more or less grouped by use, in 
the order I found them.

Note: If you think my classification is crap, and have better ideas, more 
useful for typing SDL code, please write so, I am not inflexible :-)

----- POV-Ray SDL keywords -----

--# Language directives (#xxx)

declare local include undef fopen fclose read write default version
case range break debug error warning
if ifdef ifndef switch while macro
else
end

-- Note: read & write are also modifiers
-- Note: version is also a predefined identifier
-- They are kept here because the # desambiguates it

--# Objects & CSG & Appearance (xxx {})

camera light_source light_group
object blob sphere cylinder box cone height_field julia_fractal
lathe prism sphere_sweep superellipsoid sor text torus
bicubic_patch disc mesh mesh2 polygon triangle smooth_triangle
plane poly cubic quartic quadric
isosurface parametric

union intersection difference merge

function array spline
vertex_vectors normal_vectors uv_vectors
face_indices normal_indices uv_indices

texture texture_list interior_texture
texture_map material_map image_map color_map colour_map pigment_map
normal_map slope_map bump_map density_map
pigment normal material interior finish reflection irid slope
pigment_pattern image_pattern
warp media scattering density background fog sky_sphere rainbow
global_settings radiosity photons pattern

transform looks_like projected_through contained_by clipped_by bounded_by

-- Note: cylinder is also a type of camera & light
-- Note: uv_vectors can also be written without soft braces (patch & mesh)
-- Note: reflection can be written without soft braces (only one color)
-- Note: interior is also a modifier for mandel & julia
-- Note: media & normal are also types for radiosity
-- Note: media is also an item (with 1 or 2 params) for photons

--# Types & Modifiers
--# (no params, or boolean param, or limited number of integer values)

linear_spline quadratic_spline cubic_spline
natural_spline bezier_spline b_spline
read write append
inverse open
perspective orthographic fisheye ultra_wide_angle omnimax panoramic
spherical spotlight jitter circular orient
media_attenuation media_interaction shadowless parallel
refraction collect pass_through global_lights
hierarchy sturm smooth
gif tga iff pot png pgm ppm jpeg tiff sys ttf
quaternion hypercomplex
linear_sweep conic_sweep
type all_intersections split_union cutaway_textures
no_shadow no_image no_reflection double_illuminate hollow
uv_mapping all use_index use_color use_colour no_bump_scale
conserve_energy fresnel
average agate boxed bozo bumps cells crackle cylindrical density_file dents
facets granite leopard marble onion planar quilted radial ripples spotted
waves wood wrinkles
solid use_alpha interpolate magnet noise_generator toroidal
ramp_wave triangle_wave sine_wave scallop_wave cubic_wave poly_wave
once map_type method fog_type hf_gray_16 charset ascii utf8

-- Note: jitter is also an item (with a float param) for photons

--# Modifiers & Items (xxx param)

-- Note: '<' vector param; ':' color param; '"' string param;
-- ',' several params; otherwise, integer or float param

rotate< scale< translate< matrix<
location< right< up< direction< sky<
angle look_at< aperture blur_samples focal_point< confidence variance
radius falloff tightness point_at< area_light,
adaptive fade_distance fade_power
threshold strength water_level tolerance
max_iteration precision slice,
u_steps v_steps flatness inside_vector<
accuracy max_gradient evaluate, max_trace precompute,
target ior dispersion dispersion_samples caustics
color: colour: rgb< rgbf< rgbt< rgbft< red green blue
filter transmit gray hf
fade_color: fade_colour: quick_color: quick_colour:
brick, checker, hexagon, brick_size< mortar bump_size
ambient: diffuse brilliance crand phong phong_size metallic specular
roughness reflection_exponent exponent thickness
gradient< spiral1 spiral2 agate_turb form< metric
offset df3" coords size mandel exterior julia,
control0 control1 altitude, turbulence< octaves omega lambda
repeat< flip black-hole, orientation< dist_exp major_radius
frequency phase intervals samples, ratio absorption: emission:
aa_threshold aa_level
eccentricity extinction distance turb_depth fog_offset fog_alt
width arc_angle falloff_angle
adc_bailout ambient_light: assumed_gamma irid_wavelength: number_of_waves
always_sample brigthness count error_bound gray_threshold load_file"
low_error_factor max_sample minimum_reuse nearest_count
pretrace_end pretrace_start recursion_limit save_file"
spacing gather, max_trace_level autostop expand_thresholds,

-- Note: area_light is also a type for photons
-- Note: metallic has an optional float parameter, it can be seen as a type

--# Predefined Identifiers

x y z t u v
yes no true false on off
clock clock_delta clock_on final_clock final_frame frame_number
image_height image_width initial_clock initial_frame pi version

--# Operators

+ - * / ? : & | < <= = >= > !=
( ) { } [ ]
, ;

--# Functions

-- Note: i=integer, f=float, s=string, v=vector, hyfn=hyper_function
-- I give here the parameters and result, to reuse this data in an API file.
-- For the lexer, this information must be removed.

f=abs(f)
f=acos(f)
f=acosh(f)
f=asc(s)
f=asin(f)
f=asinh(f)
f=atan(f)
f=atanh(f)
f=atan2(f, f)
f=ceil(f)
f=cos(f)
f=cosh(f)
f=defined(identifier)
f=degrees(f)
f=dimensions(array_identifier)
f=dimension_size(array_identifier, i)
f=div(f, f)
f=exp(f)
f=file_exists(s)
f=floor(f)
f=inside(solid_object_ident, v)
f=int(f)
f=ln(f)
f=log(f)
f=max(f, f [, f]...)
f=min(f, f [, f]...)
f=mod(f, f)
f=pow(f, f)
f=prod(identifier, f, f, f)
f=radians(f)
f=rand(i)
f=seed(i)
f=select(f, f, f [, f]...)
f=sin(f)
f=sinh(f)
f=sqrt(f)
f=strcmp(s, s)
f=strlen(s)
f=sum(identifier, f, f, f)
f=tan(f)
f=tanh(f)
f=val(s)
f=vdot(v, v)
f=vlength(v)
v=min_extent(object_identifier)
v=max_extent(object_identifier)
v=trace(object_identifier, v, v [, v])
v=vaxis_rotate(v, v, f)
v=vcross(v, v)
v=vrotate(v, v)
v=vnormalize(v)
v=vturbulence(f, f, f, v)
s=chr(i)
s=concat(s, s [, s]...)
s=str(f, i, i)
s=strlwr(s)
s=strupr(s)
s=substr(s, i, i)
s=vstr(i, v, s, i, i)
hyfn=sqr(f)
hyfn=cube(f)
hyfn=reciprocal(f)
hyfn=pwr(f, f)

-- 
--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/

Post a reply to this message

In article <Xns### [at] 204213191226> , Philippe Lhoste 
<Phi### [at] GMXnet>  wrote:

> I have done it manually, reading carefully the official Quick Reference and
> typing all keywords I found, trying to classify them by use, removing
> duplicates.
> Then I flattened the file (one keyword per line), sorted it and removed
> forgotten duplicates.

Hint:

<http://www.povray.org/documentation/view/135/>

Or the source code, file tokenize.cpp, of course.

    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org

Post a reply to this message

"Thorsten Froehlich" <tho### [at] trfde> wrote in
news:3d99c858$1@news.povray.org: 

> In article <Xns### [at] 204213191226> , Philippe Lhoste 
> <Phi### [at] GMXnet>  wrote:
> 
>> I have done it manually, reading carefully the official Quick Reference
>> and typing all keywords I found, trying to classify them by use,
>> removing duplicates.
>> Then I flattened the file (one keyword per line), sorted it and removed
>> forgotten duplicates.
> 
> Hint:
> 
> <http://www.povray.org/documentation/view/135/>

I knew I saw them somewhere, then I forgot it!

> Or the source code, file tokenize.cpp, of course.

Yes, I already answered in anticipation these remarks, anyway :-)
But using these resources may have lightened my work, indeed.

Thank you.

-- 
--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/

Post a reply to this message

"Philippe Lhoste" <Phi### [at] GMXnet> wrote in message
news:Xns### [at] 204213191226...
> I have built a list of POV-Ray keywords, the first step toward a Scintilla
> lexer.
> I have done it manually, reading carefully the official Quick Reference
and
> typing all keywords I found, trying to classify them by use, removing
> duplicates.
> Then I flattened the file (one keyword per line), sorted it and removed
> forgotten duplicates.
> Now the fun part: I wrote a small Lua script to parse Chaffe's quick
> reference (excellent work, very nice and clear).

Your welcome!

> My script extracted from
> the HTML file all words in bold (between <b> and </b>).
> After cleaning, I compared both files side by side in Excel.
>
> It seems I was careful enough, with only one keyword forgotten.

Keep in mind, however, that I left out of the Quick Reference some obsolete
but still acceptable syntax.  For example, the "tiles" texture and the blob
"component".

>  [snip]

I also had an idea to create a lexer and parser, using ANTLR.  Thought I
could do it until I realised that several language directives could appear
just about anywhere, even within float expressions.  Even if one wants to
attempt the evaluation of conditional expressions and the expansion of
macros in the parser or lexer itself, defining a grammar for POV-Ray is not
a trivial task.

--
Robert Chaffe
http://www.donovansweb.com/~chaffe/

Post a reply to this message

"Robert Chaffe" <rob### [at] earthlinknet> wrote in
news:3d9a6428@news.povray.org: 

> "Philippe Lhoste" <Phi### [at] GMXnet> wrote in message
> news:Xns### [at] 204213191226...
>> I have built a list of POV-Ray keywords, the first step toward a
>> Scintilla lexer.
>> My script extracted from
>> the HTML file all words in bold (between <b> and </b>).
>> After cleaning, I compared both files side by side in Excel.
>>
>> It seems I was careful enough, with only one keyword forgotten.
> 
> Keep in mind, however, that I left out of the Quick Reference some
> obsolete but still acceptable syntax.  For example, the "tiles" texture
> and the blob "component".

Well, they are not in the official quick reference either. I found them on 
the documentation, but I won't search everywhere to list them.
I would be interested by such a list, though, we could highlight them in red 
to warn they are obsolete.

>>  [snip]
> 
> I also had an idea to create a lexer and parser, using ANTLR.  Thought I
> could do it until I realised that several language directives could
> appear just about anywhere, even within float expressions.  Even if one
> wants to attempt the evaluation of conditional expressions and the
> expansion of macros in the parser or lexer itself, defining a grammar
> for POV-Ray is not a trivial task.

Well, I am not so ambitious, I just want to make a syntax highlighting lexer 
for Scintilla. Which shouldn't be a big task, as I will derivate it from the 
already existing C++ lexer.

-- 
--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/

Post a reply to this message

I have a lex/yacc grammer for C++ already and
would be interested to help create a parser for
POV-SDL.  I've written several very sophisticated
grammer parsers including one that looks alot like
POV...I have several uses for it already that are
just waiting...

Emory Stagmer

Philippe Lhoste wrote:
> 
> "Robert Chaffe" <rob### [at] earthlinknet> wrote in
> news:3d9a6428@news.povray.org:
> 
> > "Philippe Lhoste" <Phi### [at] GMXnet> wrote in message
> > news:Xns### [at] 204213191226...
> >> I have built a list of POV-Ray keywords, the first step toward a
> >> Scintilla lexer.
> >> My script extracted from
> >> the HTML file all words in bold (between <b> and </b>).
> >> After cleaning, I compared both files side by side in Excel.
> >>
> >> It seems I was careful enough, with only one keyword forgotten.
> >
> > Keep in mind, however, that I left out of the Quick Reference some
> > obsolete but still acceptable syntax.  For example, the "tiles" texture
> > and the blob "component".
> 
> Well, they are not in the official quick reference either. I found them on
> the documentation, but I won't search everywhere to list them.
> I would be interested by such a list, though, we could highlight them in red
> to warn they are obsolete.
> 
> >>  [snip]
> >
> > I also had an idea to create a lexer and parser, using ANTLR.  Thought I
> > could do it until I realised that several language directives could
> > appear just about anywhere, even within float expressions.  Even if one
> > wants to attempt the evaluation of conditional expressions and the
> > expansion of macros in the parser or lexer itself, defining a grammar
> > for POV-Ray is not a trivial task.
> 
> Well, I am not so ambitious, I just want to make a syntax highlighting lexer
> for Scintilla. Which shouldn't be a big task, as I will derivate it from the
> already existing C++ lexer.
> 
> --
> --=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--
> Philippe Lhoste (Paris -- France)
> Professional programmer and amateur artist
> http://jove.prohosting.com/~philho/

Post a reply to this message