please have a look at <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=507915>:

PoV-Ray can not use filenames allowed by the standard Debian encoding, UTF-8. As
an example I tried to include a file, which creates a mountain, called

itself works, (povray accepts it as an input file) it is not possible to have
it included:

  0:00:00 Parsing

Possible Parse Error: Non-ASCII character has been replaced by space character.

Possible Parse Error: Non-ASCII character has been replaced by space character.

Possible Parse Error: Could not find file 'monta  a.pov'

File Context (5 lines):
#include "colors.inc"
#include "camaras.inc.pov"

Parse Error: Cannot open include file monta  a.pov.

Parse Warning: Check that the file is in a directory specifed with a +L switch
 or 'Library_Path=' .INI item. Standard include files are in the include
 directory or folder. Please read your documentation carefully.

Post a reply to this message

Torsten Werner wrote:
> please have a look at <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=507915>:
> 
> PoV-Ray can not use filenames allowed by the standard Debian encoding, UTF-8. As
> an example I tried to include a file, which creates a mountain, called

> itself works, (povray accepts it as an input file) it is not possible to have
> it included:

This is not about Debian support, it is about POV-Ray Scene Description 
Language not permitting non-ASCII characters in filenames included in the 
scene file. This is intentional in POV-Ray 3.6 and it is not a defect.

	Thorsten, POV-Team

Post a reply to this message

Analysis of the code in pov_util.cpp indicates heavy reliance on 8-bit 
C-string functions when processing filenames, which would need 
nontrivial recoding to support non-ASCII filenames.

The core problem is that filenames/pathspecs are considered strings, 
when they should be promoted to their own datatype of which the string 
aspect is an implementation and/or interface detail. Code that does

   char* filename = Parse_C_string(true); // true indicates "is filename"

would become

   file_id* filename = Parse_FileID();


It's much friendler to implement file_id types in C++ (e.g., boost has a 
type which makes it easy to express multipart paths using a '/' op 
overload), so if POV 4.x is using C++, could do it there. A file_id type 
also makes it easier to encapsulate policies for path separators, drive 
letters, extensions, legal filenames and filename characters, language 
support, etc.

Ray




Torsten Werner wrote:
> please have a look at <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=507915>:
> 
> PoV-Ray can not use filenames allowed by the standard Debian encoding, UTF-8. As
> an example I tried to include a file, which creates a mountain, called

> itself works, (povray accepts it as an input file) it is not possible to have
> it included:
> 
>   0:00:00 Parsing

> Possible Parse Error: Non-ASCII character has been replaced by space character.

> Possible Parse Error: Non-ASCII character has been replaced by space character.

> Possible Parse Error: Could not find file 'monta  a.pov'

> File Context (5 lines):
> #include "colors.inc"
> #include "camaras.inc.pov"

> Parse Error: Cannot open include file monta  a.pov.

> Parse Warning: Check that the file is in a directory specifed with a +L switch
>  or 'Library_Path=' .INI item. Standard include files are in the include
>  directory or folder. Please read your documentation carefully.
> 
> 
>

Post a reply to this message

Ray Gardener wrote:
> Analysis of the code in pov_util.cpp indicates heavy reliance on 8-bit 
> C-string functions when processing filenames, which would need 
> nontrivial recoding to support non-ASCII filenames.
> 
> The core problem is that filenames/pathspecs are considered strings, 
> when they should be promoted to their own datatype of which the string 
> aspect is an implementation and/or interface detail. Code that does
> 
>   char* filename = Parse_C_string(true); // true indicates "is filename"

Actually, that line is not he problem, but the fix. POV-Ray is perfectly 
capable of parsing UTF-8, but the remaining file handling code cannot always 
deal with UTF-8 strings, and on non-recent-Unix systems it is not even 
supposed to. As inspecting the code for UTF-8 would have been overkill for 
3.6, the design decision was to disable it, which is why the line you quoted 
above enforces an ASCII string.

Either way, it is not a bug and the design is not going to be changed for 
3.6 either.

	Thorsten, POV-Team

Post a reply to this message

Isn't 3.6 already out? Or do you mean a future version of 3.6?

Ray


Thorsten Froehlich wrote:
> Ray Gardener wrote:
>> Analysis of the code in pov_util.cpp indicates heavy reliance on 8-bit 
>> C-string functions when processing filenames, which would need 
>> nontrivial recoding to support non-ASCII filenames.
>>
>> The core problem is that filenames/pathspecs are considered strings, 
>> when they should be promoted to their own datatype of which the string 
>> aspect is an implementation and/or interface detail. Code that does
>>
>>   char* filename = Parse_C_string(true); // true indicates "is filename"
> 
> Actually, that line is not he problem, but the fix. POV-Ray is perfectly 
> capable of parsing UTF-8, but the remaining file handling code cannot 
> always deal with UTF-8 strings, and on non-recent-Unix systems it is not 
> even supposed to. As inspecting the code for UTF-8 would have been 
> overkill for 3.6, the design decision was to disable it, which is why 
> the line you quoted above enforces an ASCII string.
> 
> Either way, it is not a bug and the design is not going to be changed 
> for 3.6 either.
> 
>     Thorsten, POV-Team

Post a reply to this message

Ray Gardener <ray### [at] daylongraphicscom> wrote:
> Isn't 3.6 already out? Or do you mean a future version of 3.6?

Probably "over and out" better hits the mark - I don't think anyone would still
volunteer to do any maintenance work on 3.6 ;) (or expect any maintenance work
to be done for it, for that matter)

Post a reply to this message

Hmm... looking back at the original post, I think the problem is that, 
while the error is caught, it's not properly handled, for two reasons:

1) The error messages don't say what the real problem is; the user
    might be able to deduce that a filename with non-ASCII chars is
    the problem, or he may not. Fixing the issue by relying on
    char replacement to spaces to trip a file-not-found error
    is... well, it's a hack. And the trip can fail because...

2) There may be a file in the same folder (or in whatever
    search paths exist) which already has the
    same name except it has a space instead of the extended char.
    The parser would then succeed and use that file, which could
    produce an erroneous rendering. This is a small possibility,
    I grant, but certainly not impossible. It's basically
    tantamount to POV-Ray thinking up what filename to use,
    when that should only be the user's decision.

fwiw, don't replace illegal filename chars -- just fail right at that 
point altogether (and with a "filename contains unsupported chars" message).

Ray



Thorsten Froehlich wrote:
> Ray Gardener wrote:
>> Analysis of the code in pov_util.cpp indicates heavy reliance on 8-bit 
>> C-string functions when processing filenames, which would need 
>> nontrivial recoding to support non-ASCII filenames.
>>
>> The core problem is that filenames/pathspecs are considered strings, 
>> when they should be promoted to their own datatype of which the string 
>> aspect is an implementation and/or interface detail. Code that does
>>
>>   char* filename = Parse_C_string(true); // true indicates "is filename"
> 
> Actually, that line is not he problem, but the fix. POV-Ray is perfectly 
> capable of parsing UTF-8, but the remaining file handling code cannot 
> always deal with UTF-8 strings, and on non-recent-Unix systems it is not 
> even supposed to. As inspecting the code for UTF-8 would have been 
> overkill for 3.6, the design decision was to disable it, which is why 
> the line you quoted above enforces an ASCII string.
> 
> Either way, it is not a bug and the design is not going to be changed 
> for 3.6 either.
> 
>     Thorsten, POV-Team

Post a reply to this message

Ray Gardener wrote:
> Hmm... looking back at the original post, I think the problem is that, 
> while the error is caught, it's not properly handled, for two reasons:
> 
> 1) The error messages don't say what the real problem is; the user
>    might be able to deduce that a filename with non-ASCII chars is
>    the problem, or he may not.

This is not include file specific, it is done that way for all strings. 
I.e. if you declare a string and pass that to the include directive, you 
have no way to handle it much better anyway. That one has to lok up for the 
first message to deduce the actual cause of a problem is not POV-Ray 
specific, it is common to all parsed languages...

	Thorsten

Post a reply to this message