![](/i/fill.gif) |
![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
On Sun, 31 Jan 2010 20:28:53 +0100, Warp <war### [at] tag povray org> wrote:
> Darren New <dne### [at] san rr com> wrote:
>> If you know the code number, it's some keyboard escape, like holding
>> down the alt key while typing on the numeric keypad or something.
>
> Are you sure that supports unicode and not just the system default
> character mapping (which in Windows would be that Windows extension to
> ISO-Latin-1, at least in western countries)?
On Sun, 16 Oct 2005 09:28:09 +0200, Fredrik Eriksson wrote:
>
> " Alt+<xxx>, where xxx is the decimal value of a code point, generates
> an OEM-encoded character.
> Alt+<0xxx>, where xxx is the decimal value of a code point, generates a
> Windows-encoded character.
> Alt+<+>+<xxxx>, where xxxx is the hexadecimal Unicode code point,
> generates a Unicode-encoded (UTF-16) character."
>http://www.microsoft.com/globaldev/reference/glossary.mspx
--
FE
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Warp wrote:
> Darren New <dne### [at] san rr com> wrote:
>> If you know the code number, it's some keyboard escape, like holding down
>> the alt key while typing on the numeric keypad or something.
>
> Are you sure that supports unicode and not just the system default
> character mapping
"some keyboard escape like ..."
I know you can type unicode, and I know you can type ASCII, but I don't
remember how to invoke one vs the other.
GIYF
http://www.fileformat.info/tip/microsoft/enter_unicode.htm
Basically, type numeric plus first to get unicode.
--
Darren New, San Diego CA, USA (PST)
Forget "focus follows mouse." When do
I get "focus follows gaze"?
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Fredrik Eriksson <fe79}--at--{yahoo}--dot--{com> wrote:
> > Alt+<+>+<xxxx>, where xxxx is the hexadecimal Unicode code point,
> > generates a Unicode-encoded (UTF-16) character."
If Windows takes the raw Unicode value (as entered by the user) and
encodes it in UTF-16 before passing it to the application, that would
mean that the application must understand UTF-16 as its input from (what
it perceives as) the keyboard.
Which programs actually support that? (And why UTF-16, of all possible
Unicode encoding formats?)
(An alternative to this is that the person who wrote that sentence has
no idea what he's talking about, unless I'm understanding incorrectly the
whole process involved.)
--
- Warp
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
On Sun, 31 Jan 2010 23:10:45 +0100, Warp <war### [at] tag povray org> wrote:
>
> If Windows takes the raw Unicode value (as entered by the user) and
> encodes it in UTF-16 before passing it to the application, that would
> mean that the application must understand UTF-16 as its input from (what
> it perceives as) the keyboard.
UTF-16 is the internal system encoding in all NT-based versions of
Windows. It is also the encoding in which programs receive incoming
character codes.
> Which programs actually support that?
I think most Windows programs are capable of handling Unicode these days.
Those that do not will either ignore non-ASCII characters or display them
wrong (typically as boxes or question marks).
--
FE
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Fredrik Eriksson <fe79}--at--{yahoo}--dot--{com> wrote:
> On Sun, 31 Jan 2010 23:10:45 +0100, Warp <war### [at] tag povray org> wrote:
> >
> > If Windows takes the raw Unicode value (as entered by the user) and
> > encodes it in UTF-16 before passing it to the application, that would
> > mean that the application must understand UTF-16 as its input from (what
> > it perceives as) the keyboard.
> UTF-16 is the internal system encoding in all NT-based versions of
> Windows. It is also the encoding in which programs receive incoming
> character codes.
I don't understand how that can work. Do programs convert the UTF-16
encoded characters back to raw Unicode values before using them? (Does
this perhaps happen inside the system call which is used to read the
character?)
UTF-encoded text (regardless of which UTF encoding scheme is used)
cannot easily be used directly in programs because characters cannot
be easily indexed (because UTF is a variable-length encoding scheme;
in the case of UTF-16 each character is either 2 or 4 bytes long,
depending on the Unicode value.) I have hard time believing that programs
are internally handling the text in UTF-16 format, so I'm assuming that
a decoding to raw Unicode is done first.
> > Which programs actually support that?
> I think most Windows programs are capable of handling Unicode these days.
But how do they handle the characters? Do they ask the system for a
(Unicode) character and get a raw Unicode wide char?
--
- Warp
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
On Mon, 01 Feb 2010 00:21:33 +0100, Warp <war### [at] tag povray org> wrote:
> Fredrik Eriksson <fe79}--at--{yahoo}--dot--{com> wrote:
>> UTF-16 is the internal system encoding in all NT-based versions of
>> Windows. It is also the encoding in which programs receive incoming
>> character codes.
>
> I don't understand how that can work. Do programs convert the UTF-16
> encoded characters back to raw Unicode values before using them? (Does
> this perhaps happen inside the system call which is used to read the
> character?)
>> I think most Windows programs are capable of handling Unicode these
>> days.
>
> But how do they handle the characters? Do they ask the system for a
> (Unicode) character and get a raw Unicode wide char?
When dealing with just one character at a time, the application receives
UTF-16 code points and must identify and deal with surrogates if needed.
For display purposes, one can offload some of the work on the Uniscribe
API. Mostly though, strings are passed around as a whole (i.e. as lists of
code points), in which case it "just works"; all standard widgets and
API-functions fully support supplementary characters.
For an application that does not need to deal with "exotic" alphabets
(e.g. Chinese), one can typically get away just fine with treating the
UTF-16 code points as if they were UCS-2.
--
FE
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Fredrik Eriksson <fe79}--at--{yahoo}--dot--{com> wrote:
> When dealing with just one character at a time, the application receives
> UTF-16 code points and must identify and deal with surrogates if needed.
Hmm, I'm not exactly sure what you mean by "UTF-16 code point".
According to the unicode.org glossary, a "code point" is a value in the
Unicode codespace, ie. a value in the range between 0 and 10FFFF. (Or what
I often refer to as "raw Unicode value".)
UTF-16 is a translation format between Unicode code points and bytes.
In other words, the raw unicode value is taken and encoded into a series
of bytes (2 or 4 of them, depending on the value) using a certain algorithm
(this encoding algorithm is designed to avoid any code point which has a
value larger than 127 into producing a byte with a value smaller than that).
Decoding from UTF-16 back to a Unicode code point is the reverse operation.
Thus I'm not exactly sure what you mean by "UTF-16 code point", as it
seems to be mixing the two things into one concept.
Anyways, if the Unicode-aware program requests a Unicode character from
the system, and the system returns it UTF-16-encoded, I suppose that means
that the program must decode it to a Unicode code point before it can use
it (unless it specifically handles UTF-16 strings directrly, of course).
Hmm, that sounds like a hindrance. Couldn't the system return raw Unicode
code points directly?
> For an application that does not need to deal with "exotic" alphabets
> (e.g. Chinese), one can typically get away just fine with treating the
> UTF-16 code points as if they were UCS-2.
OTOH, if a program is Unicode-aware, it should really be prepared to
handle any characters in the entire Unicode codespace.
--
- Warp
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
On Mon, 01 Feb 2010 01:13:34 +0100, Warp <war### [at] tag povray org> wrote:
>
> Hmm, I'm not exactly sure what you mean by "UTF-16 code point".
I mean a single 16-bit value from a UTF-16 string.
> Thus I'm not exactly sure what you mean by "UTF-16 code point", as it
> seems to be mixing the two things into one concept.
Yeah, I should have used "code unit" instead.
> Anyways, if the Unicode-aware program requests a Unicode character from
> the system, and the system returns it UTF-16-encoded, I suppose that
> means that the program must decode it to a Unicode code point before it
> can use it (unless it specifically handles UTF-16 strings directrly, of
> course).
Yes, unless...
> Hmm, that sounds like a hindrance. Couldn't the system return raw
> Unicode code points directly?
Starting with Windows XP, it can.
--
FE
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
Hope the following helps:
Internally .net stores characters as unicode (UTF16). There is an "encoding"
class that helps you to deal with encodings. Windows .net API provides you
with the option to use many different character encodings.
Here some excerpts from the MS online help:
Dim fileReader As String
fileReader = My.Computer.FileSystem.ReadAllText("C:\test.txt",
System.Text.Encoding.UTF32)
MsgBox(fileReader)
Writing goes something like this:
My.Computer.FileSystem.WriteAllText(file ,text ,append ,encoding)
There are a lot of different encodings supported by .net:
ASCIIEncoding
Represents an ASCII character encoding of Unicode characters.
UnicodeEncoding
Represents a UTF-16 encoding of Unicode characters.
UTF32Encoding
Represents a UTF-32 encoding of Unicode characters.
UTF7Encoding
Represents a UTF-7 encoding of Unicode characters.
UTF8Encoding
Represents a UTF-8 encoding of Unicode characters.
Chr uses the Encoding class in the System.Text namespace to determine if the
current thread is using a single-byte character set (SBCS) or a double-byte
character set (DBCS). It then takes CharCode as a code point in the
appropriate set. The range can be 0 through 255 for SBCS characters
and -32768 through 65535 for DBCS characters.
The returned value depends on the code page for the current thread, which is
contained in the ANSICodePage property of the TextInfo class in the
System.Globalization namespace. You can obtain ANSICodePage by specifying
System.Globalization.CultureInfo.CurrentCulture.TextInfo.ANSICodePage.
ChrW takes CharCode as a Unicode code point. The range is independent of the
culture and code page settings for the current thread. Values from -32768
through -1 are treated the same as values in the range +32768 through
+65535.
Numbers from 0 through 31 are the same as standard nonprintable ASCII codes.
For example, Chr(10) returns a line feed character.
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
> OK, so here's a question: How do you actually type in Unicode characters
> that aren't on your keyboard?
Hold down left alt and press some numbers on the keypad. THe only one I can
remember is the one for the plus/minus symbol (Alt+0177) as I use it a lot
and it's not on my keyboard.
> It's nice that Unicode exists, and a tiny fraction of software in
> existence even supports it,
Funny I found that most software does support it, otherwise I would be
continually screwed in my job not being able to open documents or understand
email from Japanese people. Here (Vista Business 64bit) I can even create a
file/folder in explorer with Japanese characters, then zip it with WinZip
and everything works fine.
> and approximately 3 fonts in the world have the appropriate glyphs in
> them.
I know when I get Japanese documents, they normally use a special font like
"MS Mincho", I suspect it would be a bit of a wasted effort to make every
font contain every single unicode character. There also seems to be fonts
like "Arial Unicode MS" that maybe contain all (most?) of them (it at least
contains all the Japanese characters).
Post a reply to this message
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |
|
![](/i/fill.gif) |
| ![](/i/fill.gif) |
|
![](/i/fill.gif) |