On 10/4/23 06:50, William F Pokorny wrote:
> The new bit I see is an existing, macro local, identifier changing type 
> from a 3D vector to a float - while the internal macro loops run. This, 
> of course, shouldn't happen and I still don't know why it is.

OK. I finally ran this bug down!

For any POV-Ray fork, or official release, using clipka's newer 
parser(*), change the line

	int Temp_Count=3000000;

to 	
	
	POV_LONG Temp_Count=9223372036854775807-1;

in the file parser.cpp and within Parser::Parse_RValue(...).


--- More detail for those interested.

There is code which counts the delta in the number of tokens found after 
seeing callable Identifiers / macro parameters. It involves the variable 
Tmp_Count. A variable I suspect was long ago initialized to a value 
thought to be many times larger than probable configuration values for 
TOKEN_OVERFLOW_RESET_COUNT.

The original coders made use of the token counter used for periodic 
parser status messages, rather than the global token counter, when 
calculating the delta in tokens parsed. This approach a little ugly in 
that it requires extra code for handling the reset/wrap cases which will 
frequently happen when TOKEN_OVERFLOW_RESET_COUNT is relatively small.

Christoph, while otherwise nicely re-factoring / cleaning up the older 
parser code, switched to using the global token counter straight up. 
This global counter has values which often run over/past the default 
Tmp_Count initialization of three million.

So... Once in a thousand blue moons, we call the 
Parser::Parse_RValue(...) code when the global token count is exactly 
three million and the delta in tokens found happens to be one.

On stepping on that landmine, code which should not run, does. This 
almost always results in an update to the wrong identifier type and a 
corruption of identifier associated data too.

A hideous bug. It's likely we didn't always know we'd tripped it. Only 
when the parser core dumped or stopped on some parser error was it clear 
something had gone wrong. I expect we all too often got weird parsing 
behavior or an odd image result instead. On thinking it something we 
did, we'd twiddle with the SDL. With the updated SDL and the bug would 
go away - or worse perhaps moves to another identifier with different 
end effects.

Further, where the parser didn't stop, the issue would often self heal 
the next time the identifier was redefined / updated (as in a loop) 
because the parser would realize the assigned value was indeed, say a 
vector, and not a float or whatever...

Animations - especially ones growing / changing frame to frame - are 
more likely to trip this bug simply for having more chances at it.

I was wrong that slowing down the parsing helped with this bug. I was 
simply changing the SDL parser's token counts enough to avoid it.

Bill P.


(*) - The v3.8 beta code backed off to an older v3.7 / v3.8 version of 
the parser which still used the token counter used for parser message 
updates regarding how many tokens have been parsed thus far. It should 
be OK with default builds excepting a couple VERY narrow exposures.

That said, I'd recommend all official code make the update above too! 
The v3.8 beta 1&2 code (and I'd bet most official POV-Ray versions...) 
are narrowly exposed:

- Should builders twiddle with the configuration variable 
TOKEN_OVERFLOW_RESET_COUNT in unlucky ways.

- Should the token delta count align in on an unfortunate harmonic with 
the relatively low default value of 2500 for TOKEN_OVERFLOW_RESET_COUNT. 
I think this not likely to happen in typical SDL.

- Should the type cast from the master POV_LONG (long long int) token 
count to 'int' itself cause a another problem - or still trigger this 
one in some round about way.

Post a reply to this message