OK, so I've written a program that generates a bunch of XML as output. 
If you open that in a web browser, it shows up as a simple parse tree. 
Which is OK, but a little verbose.

Fortunately, using the wonders of XSL (specifically, XSLT) you can have 
the browser transform the raw XML into a pretty-looking XHTML document.

But *only* if you edit the original XML file to mention the XSLT file 
that you want to apply.

Now, it ought to be simple to write a 3-line file that says "take this 
XML file and that XSLT file, and display the result". But after over a 
week searching the Internet and even consulting Stack Overflow, I cannot 
find any way of doing this. I literally cannot *believe* that something 
so trivially simple is also so utterly impossible.

If there /is/ a way of doing this, every single XSL tutorial in 
Christendom fails to mention it. No XSL FAQs mention it either - it's 
not even that they say it can't be done, it's that none of them even 
contain a question about this topic. This raises the alarming 
possibility that what I'm trying to do might actually be impossible 
after all. (Which is slightly daft, given that it's the entire point of 
XSL existing in the first place...)

To be clear: It seems to be perfectly possible to apply any XSL to any 
XML if you use an external 3rd-party tool. You just can't do it with a 
*web browser*. You know, those programs that know about XML and XSL and 
are installed on every computer in the world. No, you have to select and 
download a special CLI utility if you want to perform such advanced 
wizardry. *sigh*

My first thought: XSL lets you include one stylesheet inside another, so 
maybe you can include one source document inside... erm, no. No you can't.

It looks like it /might/ be plausible to do something with the XSL 
document() function - but I haven't figured out exactly how yet.

Apparently XML itself has a general include feature called XInclude. 
According to Wikipedia, "Browser support: None". *facepalm* OK, so 
that's really going to help then.

About the closest thing I've found is a website that says that if you 
copy and paste a few miles of convoluted JavaScript code, you can make a 
web page that runs the script which then loads the required XML in the 
background, lots the XSL, applies one to the other, and inserts the 
result into a specific position in the DOM of the current document. In 
particular, it cannot /replace/ the current document, it can only be 
/inserted into/ it. So if your XSL generates an entire XHTML document... 
forget it.

(I presume it goes without saying that the first thing this code snippet 
has to do is test whether it's running on Internet Explorer or a 
standards compliant browser. :-P Some things never change...)

So it looks like I'm going to have to sit down and write a special tool 
to take an XML file and copy it, inserting the correct XSL reference 
into the header. How pathetic...

Post a reply to this message

On 20/02/2012 04:21 PM, Invisible wrote:

> (I presume it goes without saying that the first thing this code snippet
> has to do is test whether it's running on Internet Explorer or a
> standards compliant browser. :-P Some things never change...)

Oh dear God...

https://bugzilla.mozilla.org/show_bug.cgi?id=98168

Short story: Using the xsl:text disable-output-escaping='yes' setting 
doesn't work. It's supposed to allow you to output arbitrary text, and 
it works with IE, but not with Firefox. And it appears they're trying to 
claim it's IE that's broken, not Firefox. >_<

Post a reply to this message

Le 20/02/2012 17:21, Invisible nous fit lire :

XML failed to have include.

> So it looks like I'm going to have to sit down and write a special tool
> to take an XML file and copy it, inserting the correct XSL reference
> into the header. How pathetic...

That's merely a shell-script:
1. find the closing of the first <?xml tag
2. insert your xsl tokens
3. copy verbatim the remaining document
4.
5. Profit.

Notice: some browser (including firefox), would:
 a. process differently a document from a http:// URL than a file:// URL
 b. not take into account XSL (as well as XSD and CSS, IIRC) not from
the same server as the original document (arguing about security!)

Post a reply to this message

Invisible <voi### [at] devnull> wrote:
> Now, it ought to be simple to write a 3-line file that says "take this 
> XML file and that XSLT file, and display the result". But after over a 
> week searching the Internet and even consulting Stack Overflow, I cannot 
> find any way of doing this. I literally cannot *believe* that something 
> so trivially simple is also so utterly impossible.

  I don't understand. This in an XML file seems to work just fine (at
least with Firefox):

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type='text/xsl' href='test.xsl'?>
...

(and then obviously you need a separate test.xsl file.)

  What exactly is the problem you are having?

  I don't know if it matters, but that test xsl file starts with:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="html" indent="no"
  doctype-public="-//W3C//DTD HTML 4.01//EN"
  doctype-system="http://www.w3.org/TR/REC-html40/strict.dtd"
 />
...

(if it makes any difference.)

-- 
                                                          - Warp

Post a reply to this message

Le 20/02/2012 18:43, Warp nous fit lire :
> (and then obviously you need a separate test.xsl file.)
> 
>   What exactly is the problem you are having?

I think I understand Invisible (ouch!):
Assume a file a.xml and another file showme_pretty.xsl

To have the browser display a.xml using showme_pretty.xsl, the a.xml
must contains the reference of showme_pretty.xsl;
There is no way to create a file a_in_pretty.xml that would
automatically combine a.xml & showme_pretty.xsl; a.xml must be copied
inside a_in_pretty.xml

Now imagine there is not ONE a.xml, but hundreds of them... Bummer!

Post a reply to this message

>> So it looks like I'm going to have to sit down and write a special tool
>> to take an XML file and copy it, inserting the correct XSL reference
>> into the header. How pathetic...
>
> That's merely a shell-script:
> 1. find the closing of the first<?xml tag
> 2. insert your xsl tokens
> 3. copy verbatim the remaining document
> 4.
> 5. Profit.

How the heck do you do that from a shell script?

...oh, wait. You probably meant a /real/ shell, not CMD.EXE, right?

Post a reply to this message

On 20/02/2012 05:43 PM, Warp wrote:
> Invisible<voi### [at] devnull>  wrote:
>> Now, it ought to be simple to write a 3-line file that says "take this
>> XML file and that XSLT file, and display the result". But after over a
>> week searching the Internet and even consulting Stack Overflow, I cannot
>> find any way of doing this. I literally cannot *believe* that something
>> so trivially simple is also so utterly impossible.
>
>    I don't understand. This in an XML file seems to work just fine (at
> least with Firefox):
>
> <?xml version="1.0" encoding="UTF-8"?>
> <?xml-stylesheet type='text/xsl' href='test.xsl'?>
> ...
>
> (and then obviously you need a separate test.xsl file.)
>
>    What exactly is the problem you are having?

Sure, that works. But now suppose I have /more than one/ XSL file. In 
order to select which one I want applied, I have to actually edit the 
XML file.

I'd prefer to have the link between XML and XSL in a file external to 
the XML file - but apparently that is impossible. (Unless you use an 
external 3rd party XSL tool.)

Post a reply to this message

Am 20.02.2012 17:58, schrieb Invisible:
> On 20/02/2012 04:21 PM, Invisible wrote:
>
>> (I presume it goes without saying that the first thing this code snippet
>> has to do is test whether it's running on Internet Explorer or a
>> standards compliant browser. :-P Some things never change...)
>
> Oh dear God...
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=98168
>
> Short story: Using the xsl:text disable-output-escaping='yes' setting
> doesn't work. It's supposed to allow you to output arbitrary text, and
> it works with IE, but not with Firefox. And it appears they're trying to
> claim it's IE that's broken, not Firefox. >_<

Actually both are standards compliant:

"An XSLT processor will only be able to disable output escaping if it 
controls how the result tree is output. This may not always be the case. 
For example, the result tree may be used as the source tree for another 
XSLT transformation instead of being output. An XSLT processor is not 
required to support disabling output escaping. If an xsl:value-of or 
xsl:text specifies that output escaping should be disabled and the XSLT 
processor does not support this, the XSLT processor may signal an error; 
if it does not signal an error, it must recover by not disabling output 
escaping."

IE behaves as mandated for a combo of an XSLT processor and an HTML 
rendering engine.

Firefox behaves as expected (by the spec authors) for a combo of an XSLT 
processor and a DOM tree rendering engine.

The spec concludes that "Since disabling output escaping may not work 
with all XSLT processors and can result in XML that is not well-formed, 
it should be used only when there is no alternative."

Post a reply to this message

Am 20.02.2012 17:21, schrieb Invisible:
> OK, so I've written a program that generates a bunch of XML as output.
> If you open that in a web browser, it shows up as a simple parse tree.
> Which is OK, but a little verbose.
>
> Fortunately, using the wonders of XSL (specifically, XSLT) you can have
> the browser transform the raw XML into a pretty-looking XHTML document.
>
> But *only* if you edit the original XML file to mention the XSLT file
> that you want to apply.

Why does this come as such a surprise to you?

Remember that the same thing is common practice for HTML and CSS as 
well: If you want the HTML to be rendered using a particular style 
sheet, you'll usually specify that in the HTML header.

I'm not sure whether all modern browsers support client-supplied CSS, 
but I wouldn't be surprised if there were still some out there that 
don't. For instance I have no idea how to choose a custom CSS in 
Firefox; you can disable all CSS, or choose which of multiple 
server-supplied CSS to use, but that seems to be about it.

Post a reply to this message

>> Oh dear God...
>>
>> https://bugzilla.mozilla.org/show_bug.cgi?id=98168
>>
>> Short story: Using the xsl:text disable-output-escaping='yes' setting
>> doesn't work. It's supposed to allow you to output arbitrary text, and
>> it works with IE, but not with Firefox. And it appears they're trying to
>> claim it's IE that's broken, not Firefox. >_<
>
> Actually both are standards compliant:

> The spec concludes that "Since disabling output escaping may not work
> with all XSLT processors and can result in XML that is not well-formed,
> it should be used only when there is no alternative."

Ah. So in fact, the W3C specification is broken, since /clearly/ a 
system should not be specified to suddenly produce completely incorrect 
output if an optional feature is not implemented.

*sigh*

Meanwhile, it looks like my only options are either to start memorising 
Unicode code points by ID, or to save my XSL file as UTF-8 and watch the 
encoding get screwed up every time I touch the file...

(Let's not even get into the question of why the standard character 
names /only/ work in HTML, and not all SGML file types...)

Post a reply to this message

>> Fortunately, using the wonders of XSL (specifically, XSLT) you can have
>> the browser transform the raw XML into a pretty-looking XHTML document.
>>
>> But *only* if you edit the original XML file to mention the XSLT file
>> that you want to apply.
>
> Why does this come as such a surprise to you?
>
> Remember that the same thing is common practice for HTML and CSS as
> well: If you want the HTML to be rendered using a particular style
> sheet, you'll usually specify that in the HTML header.

Well, what you say is of course true.

Then again, CSS just applies a few font settings to an HTML file. XSL 
transforms one file into a totally different one. If every single time 
you open that file you're always going to apply the same transformation, 
you might as well just not bother keeping the original file, and only 
store the transformed version. It completely defeats the whole purpose 
of XSL.

Actually, no, that's not /completely/ true. If several files all use the 
same XSL, then changing that one XSL file immediately changes /all/ of 
the files that use it.

Even so, XSL is clearly crying out for the ability to apply more than 
one XSL file to the same source file. And indeed you can trivially do 
that - just not with a web browser, unfortunately.

> I'm not sure whether all modern browsers support client-supplied CSS,
> but I wouldn't be surprised if there were still some out there that
> don't. For instance I have no idea how to choose a custom CSS in
> Firefox; you can disable all CSS, or choose which of multiple
> server-supplied CSS to use, but that seems to be about it.

I only recently discovered that Firefox even has a UI for changing the 
CSS. ;-) (Or turning it off, for that matter.) I have no idea whether 
you can supply your own CSS; isn't that an optional feature of the spec? 
There's probably a way to do it, but there might not be a UI. (E.g., 
there may be something hidden in about:config somewhere.)

Post a reply to this message

Le 2012-02-21 04:12, Invisible a écrit :
>>> So it looks like I'm going to have to sit down and write a special tool
>>> to take an XML file and copy it, inserting the correct XSL reference
>>> into the header. How pathetic...
>>
>> That's merely a shell-script:
>> 1. find the closing of the first<?xml tag
>> 2. insert your xsl tokens
>> 3. copy verbatim the remaining document
>> 4.
>> 5. Profit.
>
> How the heck do you do that from a shell script?
>
> ...oh, wait. You probably meant a /real/ shell, not CMD.EXE, right?

1. write the program that will do it using your favorite language 
(*cough*Haskell*cough*) and name it BAR.EXE

2. FOR %FOO in (*.xml) DO TYPE %FOO | BAR > new%FOO

Assuming BAR.EXE reads from stdin and output to stdout, this will run it 
against all the .xml files in the directory and create a second set 
called new*.xml

Alternately, find Win32 versions of awk, sed, perl, or something like that.


-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

Le 2012-02-21 05:26, Invisible a écrit :
>>> Fortunately, using the wonders of XSL (specifically, XSLT) you can have
>>> the browser transform the raw XML into a pretty-looking XHTML document.
>>>
>>> But *only* if you edit the original XML file to mention the XSLT file
>>> that you want to apply.
>>
>> Why does this come as such a surprise to you?
>>
>> Remember that the same thing is common practice for HTML and CSS as
>> well: If you want the HTML to be rendered using a particular style
>> sheet, you'll usually specify that in the HTML header.
>
> Well, what you say is of course true.
>
> Then again, CSS just applies a few font settings to an HTML file.

and positioning, and size, and background image, and whether the object 
shows up at all on a particular media, etc... but let's not let that get 
in the way of a good argument.

> XSL
> transforms one file into a totally different one. If every single time
> you open that file you're always going to apply the same transformation,
> you might as well just not bother keeping the original file, and only
> store the transformed version. It completely defeats the whole purpose
> of XSL.
>
> Actually, no, that's not /completely/ true. If several files all use the
> same XSL, then changing that one XSL file immediately changes /all/ of
> the files that use it.
>
> Even so, XSL is clearly crying out for the ability to apply more than
> one XSL file to the same source file. And indeed you can trivially do
> that - just not with a web browser, unfortunately.
>
>> I'm not sure whether all modern browsers support client-supplied CSS,
>> but I wouldn't be surprised if there were still some out there that
>> don't. For instance I have no idea how to choose a custom CSS in
>> Firefox; you can disable all CSS, or choose which of multiple
>> server-supplied CSS to use, but that seems to be about it.
>
> I only recently discovered that Firefox even has a UI for changing the
> CSS. ;-) (Or turning it off, for that matter.) I have no idea whether
> you can supply your own CSS; isn't that an optional feature of the spec?
> There's probably a way to do it, but there might not be a UI. (E.g.,
> there may be something hidden in about:config somewhere.)

You can override server-side CSS in Firefox by putting stuff in [your 
profiledir]\Chrome\userContent.css.

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

Invisible <voi### [at] devnull> wrote:
> Sure, that works. But now suppose I have /more than one/ XSL file. In 
> order to select which one I want applied, I have to actually edit the 
> XML file.

  What do you expect? For the XML file to read your mind?

-- 
                                                          - Warp

Post a reply to this message

On 21/02/2012 17:14, Warp wrote:
> Invisible<voi### [at] devnull>  wrote:
>> Sure, that works. But now suppose I have /more than one/ XSL file. In
>> order to select which one I want applied, I have to actually edit the
>> XML file.
>
>    What do you expect? For the XML file to read your mind?

No, I wanted to put that information into a separate file, that's all. 
It's not a complex idea.

Post a reply to this message

Orchid Win7 v1 <voi### [at] devnull> wrote:
> On 21/02/2012 17:14, Warp wrote:
> > Invisible<voi### [at] devnull>  wrote:
> >> Sure, that works. But now suppose I have /more than one/ XSL file. In
> >> order to select which one I want applied, I have to actually edit the
> >> XML file.
> >
> >    What do you expect? For the XML file to read your mind?

> No, I wanted to put that information into a separate file, that's all. 
> It's not a complex idea.

  Create an xml file that includes (using XInclude) the other XML file
you want as well as the xsl file?

-- 
                                                          - Warp

Post a reply to this message

Am 21.02.2012 11:09, schrieb Invisible:

>> The spec concludes that "Since disabling output escaping may not work
>> with all XSLT processors and can result in XML that is not well-formed,
>> it should be used only when there is no alternative."
>
> Ah. So in fact, the W3C specification is broken, since /clearly/ a
> system should not be specified to suddenly produce completely incorrect
> output if an optional feature is not implemented.

That's why they warn about this possible difference, so that the user 
can decide whether to (a) make sure that the XSLT file is only used with 
an XSLT processor that supports the feature, or (b) make sure that the 
output is not *completely* incorrect if the feature is unavailable.

That's not broken spec, that's just perfectly normal optional features.

> Meanwhile, it looks like my only options are either to start memorising
> Unicode code points by ID, or to save my XSL file as UTF-8 and watch the
> encoding get screwed up every time I touch the file...

Ah, so your /real/ problem is a crappy text editor.

> (Let's not even get into the question of why the standard character
> names /only/ work in HTML, and not all SGML file types...)

Because they're a HTML-specific invention?

Post a reply to this message

On 21/02/2012 08:40 PM, Warp wrote:
> Orchid Win7 v1<voi### [at] devnull>  wrote:
>> On 21/02/2012 17:14, Warp wrote:
>>> Invisible<voi### [at] devnull>   wrote:
>>>> Sure, that works. But now suppose I have /more than one/ XSL file. In
>>>> order to select which one I want applied, I have to actually edit the
>>>> XML file.
>>>
>>>     What do you expect? For the XML file to read your mind?
>
>> No, I wanted to put that information into a separate file, that's all.
>> It's not a complex idea.
>
>    Create an xml file that includes (using XInclude) the other XML file
> you want as well as the xsl file?

That's a great idea. Except that apparently no browser supports XInclude 
yet. As I pointed out in my original post.

Post a reply to this message

>> Ah. So in fact, the W3C specification is broken, since /clearly/ a
>> system should not be specified to suddenly produce completely incorrect
>> output if an optional feature is not implemented.
>
> That's why they warn about this possible difference, so that the user
> can decide whether to (a) make sure that the XSLT file is only used with
> an XSLT processor that supports the feature, or (b) make sure that the
> output is not *completely* incorrect if the feature is unavailable.
>
> That's not broken spec, that's just perfectly normal optional features.

It's a broken spec, in that it provides no way to design a transform so 
that if DEO is not implemented, the output still looks sane. There isn't 
even a way to say "sorry, the required feature is not available". 
Instead, it just prints random gibberish without telling you or giving 
you any chance to correct it.

>> Meanwhile, it looks like my only options are either to start memorising
>> Unicode code points by ID, or to save my XSL file as UTF-8 and watch the
>> encoding get screwed up every time I touch the file...
>
> Ah, so your /real/ problem is a crappy text editor.

The /real/ problem is that there is no way of knowing what encoding a 
given text file has. So it's not safe to use non-ASCII characters in a 
text file. Which is why character entities were invented in the first 
place...

>> (Let's not even get into the question of why the standard character
>> names /only/ work in HTML, and not all SGML file types...)
>
> Because they're a HTML-specific invention?

Um, why?

It's not like only HTML could possibly need non-ASCII characters.

Post a reply to this message

Am 22.02.2012 10:07, schrieb Invisible:

>>> Meanwhile, it looks like my only options are either to start memorising
>>> Unicode code points by ID, or to save my XSL file as UTF-8 and watch the
>>> encoding get screwed up every time I touch the file...
>>
>> Ah, so your /real/ problem is a crappy text editor.
>
> The /real/ problem is that there is no way of knowing what encoding a
> given text file has. So it's not safe to use non-ASCII characters in a
> text file. Which is why character entities were invented in the first
> place...

Use an editor that places a BOM at the start of UFT-8 files and bob's 
your uncle.

Post a reply to this message

>>> Ah, so your /real/ problem is a crappy text editor.
>>
>> The /real/ problem is that there is no way of knowing what encoding a
>> given text file has. So it's not safe to use non-ASCII characters in a
>> text file. Which is why character entities were invented in the first
>> place...
>
> Use an editor that places a BOM at the start of UFT-8 files and bob's
> your uncle.

In other words, "there is this informal undocumented /convention/ that 
if a file starts with a BOM [even though UTF-8 does not require such a 
mark, since there /is/ no byte order], then it is presumed to contain 
UTF-8".

I wonder how widely implemented this undocumented feature is?

Post a reply to this message

Am 22.02.2012 22:28, schrieb Orchid Win7 v1:

>> Use an editor that places a BOM at the start of UFT-8 files and bob's
>> your uncle.
>
> In other words, "there is this informal undocumented /convention/ that
> if a file starts with a BOM [even though UTF-8 does not require such a
> mark, since there /is/ no byte order], then it is presumed to contain
> UTF-8".
>
> I wonder how widely implemented this undocumented feature is?

Most likely more widespread than you think.

As an alternative, forget UTF-8 and go for UTF-16.

Post a reply to this message

>> I wonder how widely implemented this undocumented feature is?
>
> Most likely more widespread than you think.

I got the distinct impression that this is a Windows-specific 
convention. (Doesn't Linux do something strange with using environment 
variables to define the "system locale"?)

> As an alternative, forget UTF-8 and go for UTF-16.

Or that, I suppose.

This still leaves me with the problem of how to generate unusual 
characters in the first place. Typing → is pretty simple. Figuring 
out how to actually generate the arrow character is not.

Post a reply to this message

Le 23/02/2012 10:11, Invisible a écrit :
> This still leaves me with the problem of how to generate unusual
> characters in the first place. Typing → is pretty simple. Figuring
> out how to actually generate the arrow character is not.

You just need the right documentation.
Either you search it each time on the web (like "utf-8 chartable"... )

or you call it correctly, and ask for unicode (utf-8 is one
presentation, unicode is what you really want), and get to print the
parts that you need often.

See http://unicode.org/charts/

And soon you will discover that there is no single font to display all
possible unicode glyphs.

You also need a unicode-compatible editor...

Post a reply to this message

On 23/02/2012 09:39 AM, Le_Forgeron wrote:
> Le 23/02/2012 10:11, Invisible a écrit :
>> This still leaves me with the problem of how to generate unusual
>> characters in the first place. Typing→ is pretty simple. Figuring
>> out how to actually generate the arrow character is not.
>
> You just need the right documentation.

Remembering to type → is much harder than remembering to type 
→. It's also far less readable. Working out how to generate the 
actual character so you can copy and paste it into your document is far 
slower than just typing some stuff with your keyboard. (It involves 
moving your hand to the mouse, for one thing...)

> and get to print the parts that you need often.

Well, yeah, I guess that's what it comes down to. Sadly.

> And soon you will discover that there is no single font to display all
> possible unicode glyphs.

Yeah, but nobody /needs/ all possible Unicode glyphs. I'm never going to 
write stuff in Linear B or Ogham. I just hand a small handful of 
non-ASCII characters - most of which /are/ widely supported in many fonts.

> You also need a unicode-compatible editor...

I've got that. The problem isn't the editor handling Unicode, it's 
figuring out which encoding an arbitrary text file happens to use. It 
seems as soon as you use any encoding other than the Windows default 
(whatever the hell that is), things get messy, rapidly.

Post a reply to this message

Le 23/02/2012 11:57, Invisible a écrit :
> It seems as soon as you use any encoding other than the Windows default
> (whatever the hell that is), things get messy, rapidly.


On Windows, yes. Even two Windows can be messy out of their natural
country (around Redmond I believe). (e.g. at work we have some English
windows and some french ones... they default to a different code page
(that's a thing back from DOS, a time where printers had micro-switchs
to provide different mapping not only to the 127-255 range but also
inside the 33-126 range... none of which usually maps exactly to the
local code page either)

Windows choose the UTF-16 way at a time (hence the wchar in API)... but
that's double the file's size of ascii... and yet, the integration is
far from done (many code still expect char of 8 bits..)

Maybe you can have some forked resources to tag the file as being a
given encoding (forked resources for a file are its filename followed by
colon and another text... it's usually invisible and yet remain
associated with the file (yet another kludge from the West of america...
the text names the resource, the value is in the associated "file"))
Maybe you can search the web about that ?
(and find a program which allow you to handle forked resources...)
As long as the file is in a NTFS filesystem, it's safe.

Post a reply to this message

Am 23.02.2012 10:11, schrieb Invisible:
>>> I wonder how widely implemented this undocumented feature is?
>>
>> Most likely more widespread than you think.
>
> I got the distinct impression that this is a Windows-specific
> convention. (Doesn't Linux do something strange with using environment
> variables to define the "system locale"?)

 From the current XML spec:

"Entities encoded in UTF-16 MUST and entities encoded in UTF-8 MAY begin 
with the Byte Order Mark described by Annex H of [ISO/IEC 10646:2000], 
section 16.8 of [Unicode] (the ZERO WIDTH NO-BREAK SPACE character, 
#xFEFF)."

As for this being supported by editors, there are two possible cases:

(1) The editor treats UTF-8 with a leading BOM as a special encoding; in 
that case, it will strip the BOM from the character stream upon reading, 
and prepend it upon writing, so you're perfectly safe here.

(2) The editor does not expect a leading BOM in UTF-8; in that case, it 
/must/ treat it according to the Unicode standard, which explicitly 
states that the BOM is actually a perfectly valid normal character, 
which just happens to be one of the many space characters, non-breaking 
in this case, with zero width; so you're perfectly safe here as well, 
unless you accidently strip it from the very beginning of the file.

Post a reply to this message

Am 23.02.2012 11:57, schrieb Invisible:

>> You also need a unicode-compatible editor...
>
> I've got that. The problem isn't the editor handling Unicode, it's
> figuring out which encoding an arbitrary text file happens to use. It
> seems as soon as you use any encoding other than the Windows default
> (whatever the hell that is), things get messy, rapidly.

That's because figuring out the encoding of an extended-ASCII text file 
is, in fact, virtually impossible (unless you know details about the 
contents, e.g. you can recognize the encoding if you know that it's an 
HTML file), due to the fact that none of them has a standardized file 
signature. With the sole exception of UTF-8 with leading BOM, where the 
BOM character can double-feature as such a signature.

A leading BOM in UTF-8 files can cause problems with files such as shell 
scripts or C/C++ source code (because it is indeed a part of the 
character stream rather than a mere file signature; strictly speaking 
the same is actually true for UTF-16 as well); but at the beginning of 
XML files it is explicitly allowed.

Post a reply to this message

Am 23.02.2012 12:53, schrieb Le_Forgeron:
> Le 23/02/2012 11:57, Invisible a écrit :
>> It seems as soon as you use any encoding other than the Windows default
>> (whatever the hell that is), things get messy, rapidly.
>
>
> On Windows, yes. Even two Windows can be messy out of their natural
> country (around Redmond I believe). (e.g. at work we have some English
> windows and some french ones... they default to a different code page
> (that's a thing back from DOS, a time where printers had micro-switchs
> to provide different mapping not only to the 127-255 range but also
> inside the 33-126 range... none of which usually maps exactly to the
> local code page either)

Huh? I'd have expected French Windows to use Latin-1 as well. Wouldn't 
be surprised about problems with the keyboard mapping though.

Unless you're talking about the command prompt, which may indeed still 
use those old IBM codepages. (But the US-American IBM codepage and 
Latin-1 differ as well, so the only advantage the Redmondians have over 
you in this matter is that their native language doesn't need more than 
95 printable characters in the first place.)

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> (2) The editor does not expect a leading BOM in UTF-8; in that case, it 
> /must/ treat it according to the Unicode standard, which explicitly 
> states that the BOM is actually a perfectly valid normal character, 
> which just happens to be one of the many space characters, non-breaking 
> in this case, with zero width; so you're perfectly safe here as well, 
> unless you accidently strip it from the very beginning of the file.

  Does that mean that xFEFF is the zero-width nbsp in both UTF-16 and UTF-8?

  Also: If the byte order happened to be the reverse of what the editor
expects (assuming the editor does not support the BOM), wouldn't the
multi-byte characters be garbage then?

-- 
                                                          - Warp

Post a reply to this message

Le 23/02/2012 14:54, Warp a écrit :

>   Does that mean that xFEFF is the zero-width nbsp in both UTF-16 and UTF-8?
> 

Tsss... no cake for you.

0xFEFF is the UTF-16 encoding of BOM (Byte order mark). It is used to
signal endianess with UTF-16 (because 0xFFFE is not a valid utf-16,
indeed U+FFFE will never be a valid glyph).

Encoding U+FEFF in utf-8:
 * has no purpose, there is no endianess to detect for utf-8 encoding
(but it is legit to have a BOM in utf-8)
 * would be done as 3 bytes: 0xEF 0xBB 0xBF

BOM can also be useful when using UTF-32. (and other esoteric encoding
of unicode, such as utf-7, or utf-ebcdic, utf-1 (misnamed, IMHO), ... )

Notice that U+FEFF is deprecated as zero-width non breaking space.
You should use U+2060 (word joiner, zero width space non breaking),
and/or U+200B (zero width space, but breaking). At least in unicode 6.1

>   Also: If the byte order happened to be the reverse of what the editor
> expects (assuming the editor does not support the BOM), wouldn't the
> multi-byte characters be garbage then?
> 
That's the reason utf-16/32 need a BOM for automatic detection.

Post a reply to this message

Am 23.02.2012 14:54, schrieb Warp:
> clipka<ano### [at] anonymousorg>  wrote:
>> (2) The editor does not expect a leading BOM in UTF-8; in that case, it
>> /must/ treat it according to the Unicode standard, which explicitly
>> states that the BOM is actually a perfectly valid normal character,
>> which just happens to be one of the many space characters, non-breaking
>> in this case, with zero width; so you're perfectly safe here as well,
>> unless you accidently strip it from the very beginning of the file.
>
>    Does that mean that xFEFF is the zero-width nbsp in both UTF-16 and UTF-8?

If you're talking about codepoint, then obviously yes.

If you're talking about encoded byte sequence, then no; in UTF-16, it 
would be encoded as xFE xFF or xFF xFE respectively, while in UTF-8 it 
would always be encoded as xEF xBB xBF.

>    Also: If the byte order happened to be the reverse of what the editor
> expects (assuming the editor does not support the BOM), wouldn't the
> multi-byte characters be garbage then?

I would be surprised to find an editor supporting UTF-16 but not the 
BOM. As for UTF-8, the byte order for multi-byte characters (i.e. 
codepoints x0100 and above) is unambiguously defined by the standard 
(using a big-endian-ish encoding); as UTF-8 requires bit shifting 
anyway, a byte-reversed encoding would provide no benefit for 
little-endian machines and therefore doesn't exist.

Post a reply to this message

Am 23.02.2012 17:49, schrieb clipka:

> BOM. As for UTF-8, the byte order for multi-byte characters (i.e.
> codepoints x0100 and above) is unambiguously defined by the standard

Strike "x0100", replace with "x0080".

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> As an alternative, forget UTF-8 and go for UTF-16.

  UTF-16 is more compact if the text consists mostly of non-ascii
characters, especially if it contains eg. kanji symbols, hiragana, etc.
(The vast majority of Japanese kanji can be expressed with 2 bytes using
UTF-16 but require 3 bytes with UTF-8.)

  However, if the text consists mostly of ascii characters, such as
English usually does, then UTF-8 is more compact than UTF-16 (which will
basically double the size of the file).

  Support for UTF-16 is still relatively poor (although getting better).
Most modern browsers should handle it ok, though, but it requires for the
server to send the proper http header to tell the browser the encoding,
and configuring the server to do this might not be trivial. (A html file
encoded in UTF-16 will look like garbage.)

-- 
                                                          - Warp

Post a reply to this message

The trouble with the whole xml thing is that it's just another 
enterprisey BS to grind CPUs idle times.  You need an xml document, an 
xml document describing the structure of the previous document, yet 
another xml document to describe how to style the original document 
itens, perhaps a xml document describing how to transform your xml 
document into another xml document.  It's an insanely verbose and 
homogeneous pile of human and machine barely readable crap.

People resented it and thus insist on saner formats, such as CSS, JSON 
and real programming languages rather than a shitload of xml abstraction 
layers, tools and java frameworks.

Post a reply to this message

Am 23.02.2012 18:35, schrieb Warp:
> clipka<ano### [at] anonymousorg>  wrote:
>> As an alternative, forget UTF-8 and go for UTF-16.
>
>    UTF-16 is more compact if the text consists mostly of non-ascii
> characters, especially if it contains eg. kanji symbols, hiragana, etc.
> (The vast majority of Japanese kanji can be expressed with 2 bytes using
> UTF-16 but require 3 bytes with UTF-8.)
>
>    However, if the text consists mostly of ascii characters, such as
> English usually does, then UTF-8 is more compact than UTF-16 (which will
> basically double the size of the file).

I guess a factor 2 in text stream size is not a serious problem with 
today's internet bandwidths.

>    Support for UTF-16 is still relatively poor (although getting better).
> Most modern browsers should handle it ok, though, but it requires for the
> server to send the proper http header to tell the browser the encoding,
> and configuring the server to do this might not be trivial. (A html file
> encoded in UTF-16 will look like garbage.)

It didn't sound like Andy would want to retrieve the XML file from a web 
server, but rather directly from the local file system. Otherwise he 
could simply go for server-side XSLT processing.

Post a reply to this message

Am 23.02.2012 19:04, schrieb nemesis:
> The trouble with the whole xml thing is that it's just another
> enterprisey BS to grind CPUs idle times. You need an xml document, an
> xml document describing the structure of the previous document, yet
> another xml document to describe how to style the original document
> itens, perhaps a xml document describing how to transform your xml
> document into another xml document. It's an insanely verbose and
> homogeneous pile of human and machine barely readable crap.
>
> People resented it and thus insist on saner formats, such as CSS, JSON
> and real programming languages rather than a shitload of xml abstraction
> layers, tools and java frameworks.

Businesses do use it quite a lot for data exchange.

But yes, XML as a mere replacement for HTML is a rather silly thing 
(except in its incarnation as XHTML); its legitimate ecologic niche on 
the web is on the server side (if anywhere), and its native habitat is 
actually totally somewhere else.

In some sense, XML is today's CSV: A generic file or data stream format 
a human /can/ create, read and/or modify with an ASCII text editor, but 
that still follows certain clear-cut rules that it can also be evaluated 
by software; and actually just a meta-format, in the sense that the 
content of the individual data fields needs to be agreed upon separately.

Post a reply to this message

nemesis <nam### [at] gmailcom> wrote:
> The trouble with the whole xml thing is that it's just another 
> enterprisey BS to grind CPUs idle times.  You need an xml document, an 
> xml document describing the structure of the previous document, yet 
> another xml document to describe how to style the original document 
> itens, perhaps a xml document describing how to transform your xml 
> document into another xml document.  It's an insanely verbose and 
> homogeneous pile of human and machine barely readable crap.

  It's verbose, but it has one advantage over most other formats: It's
standardized and pretty well supported.

  It has many advantages over many other formats. One example is character
encoding. With all types of character encodings out there, and support
for them in different file formats and programs being what they are, a
*standardized* form for representing special characters can be really
useful. Also, any program that reads XML ought to support it regardless
of which character encoding it uses (at least if the program uses a
generic XML parser internally).

  Compare this to, for example, just a simple raw .txt file. Which encoding
does it use? ISO-Latin-1? ISO-Latin-9? UTF-8? Shift JIS? EUC-JP? ISO-2022-JP?
Something else completely? Impossible to say. With an XML file you don't have
to care. (As said, if your program uses a generic XML parser, the character
encoding used in the input XML file becomes a non-issue.)

  Not that this exact same thing wouldn't be possible with a less verbose
format, but as said, XML is widely supported so it has this implicit
advantages over many other formats.

-- 
                                                          - Warp

Post a reply to this message

Le 23/02/2012 20:06, Warp nous fit lire :
>   It's verbose, but it has one advantage over most other formats: It's
> standardized and pretty well supported.
> 

Well, XML is a container. The problem is lack of intelligent design for
the inside. It is too often the Excel sheet of today: A bunch of
entries, without consistency.

Indeed, with a bit of base64 encapsulation, you could put an BINARY
excel sheet file into a XML document. And advertise that you output XML.

And to make matter more interesting, some find it enterprisey to have
xml inside xml... and other old CSV into XML too (without reinterpreting
the data, so it's just a formatting. A dumb formatting).



>   It has many advantages over many other formats. One example is character
> encoding. With all types of character encodings out there, and support
> for them in different file formats and programs being what they are, a
> *standardized* form for representing special characters can be really
> useful. Also, any program that reads XML ought to support it regardless
> of which character encoding it uses (at least if the program uses a
> generic XML parser internally).

Read it, yes. Understand it, that's another whole story!
Same as: I can read latin or japanese in katakana, with few error on
sound. That does not means I get the meaning. At least I can edit it
like a monkey.

>   Not that this exact same thing wouldn't be possible with a less verbose
> format, but as said, XML is widely supported so it has this implicit
> advantages over many other formats.

XML is interesting when exchanging documents/data, once the big bosses
and their technical staffs have agreed on a XSD. But whenever you add a
third company, you need to negociate another XSD (with a totally
different approach of the data, not even compatible with the first one).

Post a reply to this message

On 23/02/2012 17:35, Warp wrote:
>    Support for UTF-16 is still relatively poor (although getting better).
> Most modern browsers should handle it ok, though, but it requires for the
> server to send the proper http header to tell the browser the encoding,
> and configuring the server to do this might not be trivial. (A html file
> encoded in UTF-16 will look like garbage.)

Isn't that what the HTML encoding tag is for? Or the XML encoding 
declaration?

Post a reply to this message

Am 23.02.2012 22:14, schrieb Orchid Win7 v1:
> On 23/02/2012 17:35, Warp wrote:
>> Support for UTF-16 is still relatively poor (although getting better).
>> Most modern browsers should handle it ok, though, but it requires for the
>> server to send the proper http header to tell the browser the encoding,
>> and configuring the server to do this might not be trivial. (A html file
>> encoded in UTF-16 will look like garbage.)
>
> Isn't that what the HTML encoding tag is for? Or the XML encoding
> declaration?

You can't read those until you know at least some basics about the 
encoding :-P

That's /one/ reason why XML requires UTF-16 files to start with a byte 
order mark: If the file starts with "FE FF" or "FF FE", it's either 
UTF-16, malformed XML (not starting with optional whitespace followed by 
"<?"), or indeed garbish, so UTF-16 is a safe bet in that case. 
Otherwise rely on the file format being backward compatible with ASCII, 
and treat it as UTF-8 (which is ASCII-compatible as well) until an 
encoding declaration tells you otherwise.

Post a reply to this message

On 2/23/2012 11:06, Warp wrote:
>    It's verbose, but it has one advantage over most other formats: It's
> standardized and pretty well supported.

It also has an advantage in that it's a *markup language*. That's what it's 
for. Just as it sucks for passing around arrays of floats or passing pov-ray 
data objects between clusters of renderers or encoding routing table 
updates, it does pretty darn good at things like writing help pages, user 
manuals, and so on.

Bitching that XML is poorly suited for something it was never designed to do 
is just pointless.

It's only verbose if you're not using it to pass around human-readable prose.

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

>> Isn't that what the HTML encoding tag is for? Or the XML encoding
>> declaration?
>
> You can't read those until you know at least some basics about the
> encoding :-P

Yeah. Isn't that fun? :-D

Post a reply to this message

On 23/02/2012 07:06 PM, Warp wrote:

>    It's verbose, but it has one advantage over most other formats: It's
> standardized and pretty well supported.

Well, it was that or JSON.

In my case, XML has XSLT. JSON does not possess anything comparable. 
What that means is that just by outputting my data as XML, I can easily 
transform it into something usable later. And I don't have to implement 
the transformation engine myself - just the actual transformation rules. 
(Which is still pretty non-trivial, mind you.)

(The other possibility is the unspeakably awful YAML...)

Some people /do/ use XML in some pretty inappropriate ways. (E.g., "if 
we store all our data as XML, we won't have to ask the DB admin to 
change the DB schema for us, and we can just run all the queries on the 
client-side instead".) Just because people do stupid things with XML, 
does not mean that XML is stupid.

That said, it /does/ seem overly complicated for the simple problem it's 
supposed to solve...

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> That's /one/ reason why XML requires UTF-16 files to start with a byte 
> order mark: If the file starts with "FE FF" or "FF FE", it's either 
> UTF-16, malformed XML (not starting with optional whitespace followed by 
> "<?"), or indeed garbish, so UTF-16 is a safe bet in that case. 
> Otherwise rely on the file format being backward compatible with ASCII, 
> and treat it as UTF-8 (which is ASCII-compatible as well) until an 
> encoding declaration tells you otherwise.

  OTOH, it would be quite trivial to guess that a HTML/XML file is
UTF-16-encoded: If each other byte is 0 and each other form valid
HTML/XML (at least up to the header section that specifies the encoding),
then it's a safe bet that it's UTF16-encoded.

-- 
                                                          - Warp

Post a reply to this message

Darren New escreveu:
> On 2/23/2012 11:06, Warp wrote:
>>    It's verbose, but it has one advantage over most other formats: It's
>> standardized and pretty well supported.
> 
> It also has an advantage in that it's a *markup language*. That's what 
> it's for. Just as it sucks for passing around arrays of floats or 
> passing pov-ray data objects between clusters of renderers or encoding 
> routing table updates, it does pretty darn good at things like writing 
> help pages, user manuals, and so on.
> 
> Bitching that XML is poorly suited for something it was never designed 
> to do is just pointless.

Precisely so, like implementing validators or transformers for xml docs. 
  How recursive and verbose you can get?  "Hey, Darren, have you already 
wrote that xml document to validate the xml document validator that 
validates our client xml document?"

I can't actually believe a typeful haskell supporter (I'm looking at 
Orchid) is here advocating an xml document rather than writing a typeful 
haskell program to validate data marshalling.

Post a reply to this message

On 2/24/2012 1:10, Invisible wrote:
> That said, it /does/ seem overly complicated for the simple problem it's
> supposed to solve...

What simple problem do you think it was supposed to solve?

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

On 25/02/2012 06:04, Darren New wrote:
> On 2/24/2012 1:10, Invisible wrote:
>> That said, it /does/ seem overly complicated for the simple problem it's
>> supposed to solve...
>
> What simple problem do you think it was supposed to solve?

The problem of marking up textual data.

A lot of people seem to think that if you just write some stuff in angle 
brackets, that's XML. But it' actually far more complicated than that.

(As an aside, those aren't angle brackets. They're inequality signs. :-P 
But I digress...)

For example, to say "this XML file actually contains XHTML", you're 
supposed to add something like 5 separate headers, each about 25 million 
miles long, to the top of the file. Why? Would one header not be sufficient?

Then there's all that stupidity with character entities. It should be 
possible to include character entities in /all/ XML documents, not just 
XHTML. But to this day, I have never discovered a way of doing this.

One of the nice ideas of XML is that you can use namespaces to include 
one sort of XML document inside enought - e.g., you could put an SVG 
file inside an XHTML file. But nooo, apparently you need a specialised 
DTD to do that. In fact, you need one DTD for XHTML + SVG, another, 
different DTD for XHTML + MathML, a third, different DTD for XHTML + 
MathML + SVG, and... ARE YOU PEOPLE INSANE?!

In short, most people just end up sticking stuff in angle brackets and 
calling that XML. /Actual/ XML is far too complicated.

Post a reply to this message

On 2/25/2012 2:37, Orchid Win7 v1 wrote:
> For example, to say "this XML file actually contains XHTML", you're supposed
> to add something like 5 separate headers, each about 25 million miles long,
> to the top of the file. Why? Would one header not be sufficient?

Really? What do you have to add? I suspect you exaggerate a bit.

> Then there's all that stupidity with character entities. It should be
> possible to include character entities in /all/ XML documents, not just
> XHTML. But to this day, I have never discovered a way of doing this.

You have to declare the entities. Otherwise, you run across &frump; and 
don't know what it means. That's what DTDs are for.

> One of the nice ideas of XML is that you can use namespaces to include one
> sort of XML document inside enought - e.g., you could put an SVG file inside
> an XHTML file. But nooo, apparently you need a specialised DTD to do that.

You can. Of course, your various processors have to know what the various 
namespaces *mean*, so you have to declare them and you have to have a 
processor that understands them (which is where the DTDs come in).

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

On 26/02/2012 12:35 AM, Darren New wrote:
> On 2/25/2012 2:37, Orchid Win7 v1 wrote:
>> For example, to say "this XML file actually contains XHTML", you're
>> supposed
>> to add something like 5 separate headers, each about 25 million miles
>> long,
>> to the top of the file. Why? Would one header not be sufficient?
>
> Really? What do you have to add? I suspect you exaggerate a bit.

First, you must say

<?xml version="1.0" encoding="ISO-8859-1" ?>

Which is fair enough, although the 5-digit ISO code number isn't exactly 
easy to remember. Then you have to say

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

Yeah, I'm really going to remember that without looking it up. :-P No, 
you can't just say "this is XHTML". You really do have to write all that 
crap. And after /that/, you still have to say

<html xmlns="http://www.w3.org/1999/xhtml">

Um, didn't I just declare this as XHTML already? At least this URL is a 
bit shorter, I guess...

In short, that's a /hell/ of a lot of typing, before I've even written 
my document. It's also a lot of typing that I have to go look up on 
three different websites every single damned time I want to write XHTML.

>> Then there's all that stupidity with character entities. It should be
>> possible to include character entities in /all/ XML documents, not just
>> XHTML. But to this day, I have never discovered a way of doing this.
>
> You have to declare the entities. Otherwise, you run across &frump; and
> don't know what it means. That's what DTDs are for.

So why isn't there a DTD somewhere that contains all the entities that 
everybody uses, which I can just link to? More to the point, since 
everybody uses the same set of entities, why aren't they built in?

>> One of the nice ideas of XML is that you can use namespaces to include
>> one
>> sort of XML document inside enought - e.g., you could put an SVG file
>> inside
>> an XHTML file. But nooo, apparently you need a specialised DTD to do
>> that.
>
> You can. Of course, your various processors have to know what the
> various namespaces *mean*, so you have to declare them and you have to
> have a processor that understands them (which is where the DTDs come in).

You would /think/ you could just say "hey, this stuff here is in the 
XHTML namespace, this stuff here is in the SVG namespace", and that 
would work. But it doesn't. You need to use a DTD specially written for 
the exact combination of XHTML+SVG. To me, this indicates that the 
system is fundamentally broken. It should not be necessary to write a 
separate DTD for every possible combination of XML documents imaginable. 
You should just be able to refer to the DTD for XHTML and the DTD for 
SVG and have it work.

Post a reply to this message