The Turkish İ Problem and Why You Should Care (2012)

I disagree with roughtly all of this.

First of all, 'ß' was a ligature -- a long time ago. It is a letter today. Disassembling it according to its original construction makes no sense today for any kind of argument about typesetting or Unicode. Further, 'ſ' is not used today in German at all, except for meta discussions like this or to stress how things used to be spelled. It makes no sense to mention it unless you are talking about font design or historic use of German (and other languages, for that matter).

Also, if you do mention it for the sake of talking about font design, in Latin fonts, 'ſs' is actually the basis for the design of 'ß', not 'ſz' -- that was mainly done in Blackletter/Fraktur when the 'z' looked different, maybe a bit like 'ʒ' (I used Unicode's'ezh' here hoping it looks right) so that old style 'ß' looks like a ligature of 'ſʒ'. This can still be seen occasionally, e.g., on Berlin street name signs. It is obsolete for most fonts today (although I quite like it).

Moreover, there is an upper case letter for 'ß': 'ẞ'. And it has existed in fine typology way before being adopted into Unicode. Actually, it's existence was probably the reason why it is now in Unicode. The official German rules are now: either use 'SS' or 'ẞ' for uppercase 'ß'. Most Germans probably do not even know that 'ẞ' exists as a choice today, although it was used on 'DER GROẞE DUDEN' even before Unicode existed.

And finally, how a glyph is designed is not necessarily decided on whether historic parts of an ancient ligature had upper case variants. So that 'ſ' has no upper case equivalent is irrelevant for both Unicode and type design.

But as a font designer or anything else, you can protest. No problem. Everyone has the right to protest. But please don't spill the Internet with wrong information, as there is enough of it already.

And I don't think 'SS'<->'ß' is similar to the Turkish 'I with/without dot' problem, because the default Unicode mapping for 'ß' is correct in all languages, while the Turkish (and also Azerbaijani) problem is correct or broken depending on language setting. This is way more problematic because an assumed universal equivalence does not hold. And you need to carefully distinguish whether a string is language specific or not, e.g., path names or IDs in data bases, etc.

> It's an absolute eye saw

I assume you mean "eyesore"

U+07E6 probably should be rendered as exactly like “SS” (not a ligature, and as a double-width character in monospace fonts). Inventing a separate glyph for it seems a bit silly and only hinders adoption. Even if it's a ligature like §, that issue won't go away. And there are design choices that are even worse than § due to historic precedent.

> As a hypothetic analogy, imagine a ‘ll’ ligature, as in ‘fallacy’, becoming an English letter – by some twist of history.

You could instead consider 'W', a non-hypothetical case of just that. Would you similarly abandon current English orthography in your font design in defense of some historical trivia?

[flagged]

It might seem like an overreaction from a western point of view, but the accusations in the context of Central Asian culture is something so extremely sensitive that people from all walks of life, from nobility to the poor kill and die over it. It is just a different frame of mind.

  The problem was that Emine's cell phone was not localized properly for Turkish and did not have the letter <ı>; when it displayed Ramazan's message, it replaced the <ı>s with <i>s.

They are two different vowels. There are 8 vowels in Turkish. All letters correspond to a different phonetic sound in the language. So you can read it as it is written. Iı sounds like e in ‘the’, and ‘i’ sounds like ‘ee’ in deep. They are different sounds, different vovels hence conveniently different letters. Difference changes all the meaning. For instance “sınır” means boundary and “sinir” means nerve. I think it is simple and brilliant design for Turkish. I respect the design choices of all the other languages but this is what works best for our own language. You can simply accept that it is a different (modified latin) alphabet for a different language.

It's not really a worse choice than reusing the same symbol to represent multiple vowels. The preceding sentence has three different vowel sounds associated with the letter e, for example (four if you include the silent -e like in choice).

This is the same as saying we have four o's in English: o, O, q, Q. Two with a tail, two without.

But you're comparing two cases of one letter to two cases of two letters. Of course two letters have, in aggregate, more glyphs than one letter.

that's like saying l and I are the same letters because they look similar in some sans serif fonts.

[translated from latin]

So while we have two i's, they have four

1. i

2. I

No, we don't have four i's in English, I(i) and J(j) are two separate letters

1. i

2. I

3. j

4. J

--> 4

I know nothing of Thai but in Hindi (devanagari) ignoring standalone vowels (अ a) they come after a consonant to modify its pronunciation, regardless of whether the mark is written predominantly before or after or below. E.g. for the consonant k क् (the mark below there indicates no vowel yet):

    क ka (kuh)
    का kā (kaa)
    कि ki (kih)
    की kī (kee)
    के ke (kay)
    कै kai (keh)
    को ko (koh)
    कौ kau (kaw)
    कु ku (koo)
    कू kū (kooo)
 

Turkish was explicitly based on the Latin script by Atatürk. If Erdogan gets his way, it'll be reverted.

Oh, I don't disagree. Some languages clearly got better treatment by the Unicode committee than others.

Nope, it’s because of how the legacy encodings were handled. See my sibling comment to yours.

C doesn't have a string data type, let alone a locale-aware one. No, the Microsoft LPCWSTR madness doesn't count.

Again, I'm now talking about "known" text in the programming context. Text that is neither user-input, or presented to a user. (E.g. column names. builtin function names in my toy spreadsheet. Whatever).

Obviously for text that is both user-input and then displayed back again to users, you are in the other category. Apart from protecting against rendering mishaps and security etc, you probably just want to preserve what they write.

But that was my point: 90% or more of the text you do is likely in the first category. And very rarely do you even have do deal with text in the second category.

The schema I meant was created in-house. I meant the tags and attribute names (schema) I can control.

Just like the compiler controls what can go in a function name (but not in a string literal or comment).

I've actually been responsible some 10 years ago for introducing a bug like this in an official release of an industry-standard tool for a somewhat niche industry. Some SQL queries we were generating ended up saying `SELECT x FROM t WHERE x < 1,02` if run on an any system with commas as the decimal separator. We found it and fixed a few weeks later, and I don't think we've ever had a complaint from the field about this, but it was still pretty eye opening about locales.

The extra irony is that me and my colleagues live in a country that actually has this kind of locale, but no one in the entire extended team was using it, everyone uses a US locale.

I think that in my country it's “1'234,5“. It was when I learned to write many years ago before computers were common.

I've had to adust to/ accomodate the difference between 1. and 1, almost daily.

Very expensive if you fuck up. Very embarrassing if you fuck up, too.

> "i" key is interpreted as English "i" when in English locale, as a special unicode character when in Turkish locale. This would be a nightmare as you would then always have to be conscious of your locale.

Isn’t this already the case with other languages? For instance, the same key on the keyboard produces a semicolon (;) in English and a Greek question mark (;) in Greek. These are distinct characters that are rendered the same (and also an easy way to troll a developer who uses an editor that doesn’t highlight non-ASCII confusables).

> 1. Have two "i" characters on Turkish keyboards, one to use when writing in English, one in Turkish. Sounds difficult to get used to. Always need to be conscious about whether writing an "English i", or a "Turkish i".

But you have to do that anyway to be able to produce the correct capitalized version: an "English I" or a "Turkish İ".

Ah, that's because I thought turks and azerbaijanis just switched keyboard layouts to type in english and to type in their native language.

What were the other possible trade-offs? I don't really see how lack of round-trip compatibility is worse than what we have now. It's breaking the whole idea of Unicode code points and for what.

How would separate code point break round-tripping specifically?

Yep I'm aware

Yes - the problem is that "i" and "I" are standard ASCII characters, while the dotted I and the dotless i are not. Creating special "Turkish I" and "Turkish i" characters would have been an alternative, but would have had its own issues (e.g. documents where only some "i"s are Turkish and the rest "regular" because different people edited it with different software/settings).

Irish script traditionally used a dot-less "i", something that persists in current road signage (anecdotally to save confusion with "í", or with adjacent old-style dotted consonants, I can't find a definitive source to cite). It's only an orthographic/type thing, it's semantically an "i", though the Unicode dot-less "i" is sometimes used online to represent it.

Is it? That's weird, I can't find the code for Latin Small Letter Dotted I. There is a Cyrillic dotted I, but that one doesn't have the dot in capitalised form.

What sebstefan is asking for is a Unicode character which is the non-capitalised form of Latin Capital Letter I With Dot Above (U+0130) which always gets capitalised to U+0130 and which U+0130 gets downcased to.

And DELETE DOT ABOVE would wnd that locale dependency.

It wouldn't matter

There's already this problem for cyrillic 'e' and latin 'e' and hundreds of other characters

People use it to create lookalike URLs and phish people

https://www.pcmag.com/news/chrome-blocks-crafty-url-phishing...

> But having it on a character as common as "i" would feel specially hellish to me.

https://en.wikipedia.org/wiki/Dotted_I_(Cyrillic)