Dang, there should be a `povray.rants` newsgroup, where it is clear from 
the get-go that any post there is to blow off steam, not step on 
anyone's toes. Anywhere, here I go.


In the wake of the tokenizer changes, I needed to clean up the internal 
handling of character encoding.

In the wake of the character encoding cleanup, I needed to make changes 
to the TrueType font handling.

In the wake of changing the TrueType font handling, I found out that the 
fonts we're currently shipping with POV-Ray are crap. Total and utter 
crap. Not from an artistic point (that could be debated), but from 
conceptual and technical points. Who made these pieces of digital 
bullshit anyway?


Let's start with a look at the choice of fonts: We have a serif font 
(`timrom.ttf`) and a sans-serif (`crystal.ttf`) font. So far, so good. 
But when we look beyond ASCII characters, pretty much all we have is in 
a third font (`cyrvetic.ttf`), and it is... no, not Unicode, that would 
be too much to expect... no, not the most widespread non-Unicode 
extension to ASCII, namely Windows-1252... no, it's Cyrillic. I mean 
come on, seriously? You can give us "дерьмо", but not good old German 
"Scheiße", despite some key developers having been German speakers? Or 
"flûte", pardon my French, despite a great tradition of French-speaking 
community members?

When we look more closely, things get even more out of whack: The 
`timrom.ttf` font _does_ go beyond the ASCII character sets; but what do 
we find? Punctuation. A whole bunch of punctuation, most of which you'll 
never ever need. But German or French letters? Not a single one in 
sight. Flûte.

And with that, we come to the technical side. And there the Scheiße is 
really am Dampfen.


Now the TrueType font format provides multiple different ways to map 
numeric values (character codes) to glyphs (the actual shapes of the 
characters), stored in so-called "CMAP tables". Each CMAP table has its 
own peculiarities.

For instamce, CMAP table (1,0) is intended to be used on classic Mac OS 
systems to map 8-bit character codes to glyphs according to the "Mac OS 
Roman" character set, while CMAP table (3,1) is intended to be used on 
Microsoft systems to map 16-bit character codes to glyphs according to 
the UCS-2 character set (essentially the 16-bit subset of Unicode).

But if we translate Unicode characters first to Mac OS Roman and then 
try to map them via CMAP table (1,0), we get Schei§e. No, not Scheiße, 
that would be fine. Schei§e with a § as in German legal paragraph. Why? 
Because in this font CMAP table (1,0) is not arranged according to MAC 
OS Roman as it should be, but according to Windows-1252. Flûte encore! 
Well, at least CMAP table (3,1) is good.

It's no surprise that CMAP table (1,0) of `cyrvetic.ttf` isn't any 
better. Of course at this point we don't expect it to be MAC OS Roman as 
the table IDs advertise. After browsing plenty of character set tables, 
I figured it's Windows-1251. But this font can do worse: Expecting CMAP 
table (3,1) to use Unicode mapping? Hah! All you get is ScheiЯe. I mean, 
flыte. Ah, дерьмо! In a table that should be arranged according to 
Unicode, and provides ample space for it, you find a Windows-1251 
mapping! Even if you shove in the Unicode values of Cyrillic characters, 
you get nothing useful out of it.


So TL;DR: The choice of available characters leaves much to be desired. 
The `timrom.ttf` and a sans-serif `crystal.ttf` are technically broken. 
But the `cyrvetic.ttf` takes the cake: Buffing POV-Ray to do proper 
Unicode handling will make that font TOTALLY useless(*).


What an absurdly crappy set of fonts!


I thknk it is time we got us some new ones to ship with POV-Ray.

Also, I have decided to try changing the font handling from our own 
dated code to using the FreeType library. That way I won't be tempted to 
work around such crap. Them old fonts no longer work? Tough luck. If the 
lib won't handle them, then maybe, just maybe, the fonts really are the 
pieces of shit I'm calling them.

(Actually I was already toying with the idea of delegating TrueType font 
file handling to a 3rd party library for other reasons; the problems 
with our old fonts are just the last straw.)


(*Yeah, I know it can still serve as a sans-serif ASCII font; but we 
already have `crystal.ttf` for that.)

Post a reply to this message

On 11-1-2019 4:01, clipka wrote:
> [snip]
> 
> 
> What an absurdly crappy set of fonts!
> 
> 
> I thknk it is time we got us some new ones to ship with POV-Ray.
> 
> [snip]

LOL; sorry Christoph; not laughing at you... I truly appreciate your 
efforts and am in awe from your dedication and thoroughness.

Just plain (historical) curiosity, but it would be interesting to 
document the path of how this came about to be. The choices of people in 
the past are often enlightening. However, do not throw time away with 
that; it is more of an archaeological value, when somebody will start 
writing the history of POV-Ray.

-- 
Thomas

Post a reply to this message

I can tell you how this former German developer solved it: The claasic Mac
version used to have special code to extract Mac Roman fonts from the system...

clipka <ano### [at] anonymousorg> wrote:
> Dang, there should be a `povray.rants` newsgroup, where it is clear from
> the get-go that any post there is to blow off steam, not step on
> anyone's toes. Anywhere, here I go.
>
>
> In the wake of the tokenizer changes, I needed to clean up the internal
> handling of character encoding.
>
> In the wake of the character encoding cleanup, I needed to make changes
> to the TrueType font handling.
>
> In the wake of changing the TrueType font handling, I found out that the
> fonts we're currently shipping with POV-Ray are crap. Total and utter
> crap. Not from an artistic point (that could be debated), but from
> conceptual and technical points. Who made these pieces of digital
> bullshit anyway?
>
>
> Let's start with a look at the choice of fonts: We have a serif font
> (`timrom.ttf`) and a sans-serif (`crystal.ttf`) font. So far, so good.
> But when we look beyond ASCII characters, pretty much all we have is in
> a third font (`cyrvetic.ttf`), and it is... no, not Unicode, that would
> be too much to expect... no, not the most widespread non-Unicode
> extension to ASCII, namely Windows-1252... no, it's Cyrillic. I mean
> come on, seriously? You can give us "дерьмо", but not good old German
> "Scheiße", despite some key developers having been German speakers? Or
> "flûte", pardon my French, despite a great tradition of French-speaking
> community members?
>
> When we look more closely, things get even more out of whack: The
> `timrom.ttf` font _does_ go beyond the ASCII character sets; but what do
> we find? Punctuation. A whole bunch of punctuation, most of which you'll
> never ever need. But German or French letters? Not a single one in
> sight. Flûte.
>
> And with that, we come to the technical side. And there the Scheiße is
> really am Dampfen.
>
>
> Now the TrueType font format provides multiple different ways to map
> numeric values (character codes) to glyphs (the actual shapes of the
> characters), stored in so-called "CMAP tables". Each CMAP table has its
> own peculiarities.
>
> For instamce, CMAP table (1,0) is intended to be used on classic Mac OS
> systems to map 8-bit character codes to glyphs according to the "Mac OS
> Roman" character set, while CMAP table (3,1) is intended to be used on
> Microsoft systems to map 16-bit character codes to glyphs according to
> the UCS-2 character set (essentially the 16-bit subset of Unicode).
>
> But if we translate Unicode characters first to Mac OS Roman and then
> try to map them via CMAP table (1,0), we get Schei§e. No, not Scheiße,
> that would be fine. Schei§e with a § as in German legal paragraph. Why?
> Because in this font CMAP table (1,0) is not arranged according to MAC
> OS Roman as it should be, but according to Windows-1252. Flûte encore!
> Well, at least CMAP table (3,1) is good.
>
> It's no surprise that CMAP table (1,0) of `cyrvetic.ttf` isn't any
> better. Of course at this point we don't expect it to be MAC OS Roman as
> the table IDs advertise. After browsing plenty of character set tables,
> I figured it's Windows-1251. But this font can do worse: Expecting CMAP
> table (3,1) to use Unicode mapping? Hah! All you get is ScheiЯe. I mean,
> flыte. Ah, дерьмо! In a table that should be arranged according to
> Unicode, and provides ample space for it, you find a Windows-1251
> mapping! Even if you shove in the Unicode values of Cyrillic characters,
> you get nothing useful out of it.
>
>
> So TL;DR: The choice of available characters leaves much to be desired.
> The `timrom.ttf` and a sans-serif `crystal.ttf` are technically broken.
> But the `cyrvetic.ttf` takes the cake: Buffing POV-Ray to do proper
> Unicode handling will make that font TOTALLY useless(*).
>
>
> What an absurdly crappy set of fonts!
>
>
> I thknk it is time we got us some new ones to ship with POV-Ray.
>
> Also, I have decided to try changing the font handling from our own
> dated code to using the FreeType library. That way I won't be tempted to
> work around such crap. Them old fonts no longer work? Tough luck. If the
> lib won't handle them, then maybe, just maybe, the fonts really are the
> pieces of shit I'm calling them.
>
> (Actually I was already toying with the idea of delegating TrueType font
> file handling to a 3rd party library for other reasons; the problems
> with our old fonts are just the last straw.)
>
>
> (*Yeah, I know it can still serve as a sans-serif ASCII font; but we
> already have `crystal.ttf` for that.)

Post a reply to this message

Am 11.01.2019 um 09:10 schrieb Thorsten Froehlich:
> I can tell you how this former German developer solved it: The claasic Mac
> version used to have special code to extract Mac Roman fonts from the system...

I intend to do that for the Windows version, too.

But I think we should have some baseline collection of fonts (if only 
for the Benchmark scene), and I'd like them to be at least reasonably 
ok. It's not like there aren't any free fonts out there we could "adopt".

Post a reply to this message

Le 11/01/2019 à 14:57, clipka a écrit :
> Am 11.01.2019 um 09:10 schrieb Thorsten Froehlich:
>> I can tell you how this former German developer solved it: The claasic
>> Mac
>> version used to have special code to extract Mac Roman fonts from the
>> system...
> 
> I intend to do that for the Windows version, too.
> 
> But I think we should have some baseline collection of fonts (if only
> for the Benchmark scene), and I'd like them to be at least reasonably
> ok. It's not like there aren't any free fonts out there we could "adopt".

Seconded.

We need a Serif, a Sans Serif and a Mono spaced, IMHO.

And good looking ones, with latin-1 page and other cool stuff.

As long as only 16 bits are parsed, the chinese glyphes are out of scope.

Post a reply to this message

Am 11.01.2019 um 17:17 schrieb Le_Forgeron:

> We need a Serif, a Sans Serif and a Mono spaced, IMHO.

Agreed.

> And good looking ones, with latin-1 page and other cool stuff.

Sure.

> As long as only 16 bits are parsed, the chinese glyphes are out of scope.

No, they're not. Not the most important ones at any rate: Roughly 30k 
CJK ideographs are assigned to the Basic Multilingual Plane.

We will of course miss out on the approx. 60k /additional/ CJK 
ideographs in Plane 2. Also, no Phoenician, Egyptian Hieroglyphs, 
Cuneiform or the like (Plane 1).

Post a reply to this message

Am 11.01.2019 um 04:01 schrieb clipka:

> Also, I have decided to try changing the font handling from our own 
> dated code to using the FreeType library. That way I won't be tempted to 
> work around such crap. Them old fonts no longer work? Tough luck. If the 
> lib won't handle them, then maybe, just maybe, the fonts really are the 
> pieces of shit I'm calling them.

Work is progressing. And it looks like I'll be ripping out the text 
primitive entirely.


Well, /almost/ entirely that is. The parser will still understand the 
`text` syntax. But instead of implementing it via a dedicated geometric 
shape it will map it to a union of `prism` objects. (Theoretically a 
single `prism` object would do in typical cases, but turns out to be 
noticeably inferior in terms of performance.)

Under the hood, the text primitive has essentially been duplicating 
prism functionality all along, except that it is limited to 2rd order 
splines. While that provides some benefits in terms of performance, it 
also means it's not up-to-date to team up with the FreeType library, 
which supports various non-TrueType font file formats, including some 
which are based on 3rd order splines.

So rather than investing any time and effort into refurbishing the 
somewhat redundant text primitive, I've instead invested some time and 
effort to see if it can be replaced with the prism primitive entirely. 
So far it seems as easy as eating pancakes.

Post a reply to this message

in news:5c3d53b6$1@news.povray.org clipka wrote:

> So far it seems as easy as eating pancakes.

Would it also be possible to make the fonts avalable as splines? Thinking 
of spheresweeps, sweeping spheres, lofting and extruding mesh's,

ingo

Post a reply to this message

On 1/14/19 10:29 PM, clipka wrote:
> Am 11.01.2019 um 04:01 schrieb clipka:
> 
>> Also, I have decided to try changing the font handling from our own 
>> dated code to using the FreeType library. That way I won't be tempted 
>> to work around such crap. Them old fonts no longer work? Tough luck. 
>> If the lib won't handle them, then maybe, just maybe, the fonts really 
>> are the pieces of shit I'm calling them.
> 
> Work is progressing. And it looks like I'll be ripping out the text 
> primitive entirely.
> 
> 
> Well, /almost/ entirely that is. The parser will still understand the 
> `text` syntax. But instead of implementing it via a dedicated geometric 
> shape it will map it to a union of `prism` objects. (Theoretically a 
> single `prism` object would do in typical cases, but turns out to be 
> noticeably inferior in terms of performance.)
> 
> Under the hood, the text primitive has essentially been duplicating 
> prism functionality all along, except that it is limited to 2rd order 
> splines. While that provides some benefits in terms of performance, it 
> also means it's not up-to-date to team up with the FreeType library, 
> which supports various non-TrueType font file formats, including some 
> which are based on 3rd order splines.
> 
> So rather than investing any time and effort into refurbishing the 
> somewhat redundant text primitive, I've instead invested some time and 
> effort to see if it can be replaced with the prism primitive entirely. 
> So far it seems as easy as eating pancakes.

And no doubt less sticky. :-)

While updating the text primitive, are you giving any thought to right 
to left and vertical syntax in addition to the existing left to right?

Aside: My recollection from looking at the text object back when I was 
playing with the hard and soft objects is that certain font-encodings 
support overlapping polygon regions by being strict about the 2d winding 
order of the path outlines for the definition of inside and outside as 
opposed to the crossing test we use. Perhaps some kind of crossing test 
compatible normalization is necessary if supporting what FreeType 
supports? Perhaps too an issue for later as I have no idea how common 
overlapped encoding is...

Bill P.

Post a reply to this message

On 1/15/19 1:38 AM, ingo wrote:
> in news:5c3d53b6$1@news.povray.org clipka wrote:
> 
>> So far it seems as easy as eating pancakes.
> 
> Would it also be possible to make the fonts avalable as splines? Thinking
> of spheresweeps, sweeping spheres, lofting and extruding mesh's,
> 
> ingo
> 

Supposing you know font splines are outline/edge based so some extra 
processing would be needed for a stroke spline as might come from a 
mouse or tablet.

Are you asking for the font splines as is or as variable width pen 
strokes say?

Aside:
-----
During a crazy few days a couple years ago I made a serious start on 
software for creating a POV-Ray library (spheresweep like def) 
equivalent of the GNU Unifont:

https://en.wikipedia.org/wiki/GNU_Unifont

The idea was to create some initial set of centered linear bezier 
segments - with start/end width spec - from the Unifont/other open 
source mono-spaced fonts with some automation and then refine it over time.

The thought was to have available a centered spline definition of high 
resolution which might be much more useful to folks than the Unifont is. 
In creating other fonts, doing character recognition or whatever. 
POV-Ray as an actual font editor/analyzer of sorts, I guess, working 
from an initial Unifont-like POV-Ray compatible center-line stroke 
database.

I am, on odd and even days, not that sane.

Bill P.

Post a reply to this message