Darren New wrote:
> Orchid XP v8 wrote:
>> The way you said "take each token and pass it through a macro" made it
>> sound like the input stream gets tokenised first
>
> It gets tokenized one token at a time. Otherwise, how would your "macro"
> be able to read the input? I'm not saying "it looks at the
> already-parsed input". I'm saying "it invokes the read() function on
> stdin to consume input that was sent to the compiler". The funky syntax
> is never seen by the tokenizer (in FORTH and LISP).
Tokenising isn't parsing; it's just the initial part where you split the
input into tokens. Thing is, each language has different ideas about
what a "token" is...
Obviously I'm never going to use Lisp or Forth, never mind macros. I'm
just mildly curious to know whether you can implement your own
tokenising rules or not.
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|