POV-Ray : Newsgroups : povray.beta-test : New version of new tokenizer : Re: New version of new tokenizer Server Time
27 Nov 2022 19:52:34 EST (-0500)
  Re: New version of new tokenizer  
From: Thorsten Froehlich
Date: 1 Jun 2018 17:20:01
Message: <web.5b11b74a905c8c8f535efa580@news.povray.org>
clipka <ano### [at] anonymousorg> wrote:
> - The v3.7 implementation simply swallowed any contiguous sequence of
> non-ASCII bytes at the start of a scene file, and just /presumed/ them
> to be an UTF-8 signature BOM. The v3.8.0-x.tokenizer implementation
> actually checks whether the non-ASCII byte sequence matches the UTF-8
> signature BOM.

This was actually a feature. Originally it checked, but it turned out that at
least at the time several editors created incorrect BOMs...

Thorsten


Post a reply to this message

Copyright 2003-2021 Persistence of Vision Raytracer Pty. Ltd.