[PATCH][2.5] UTF-8 support in console

Chris Heath (chris@heathens.co.nz)
Sat, 31 May 2003 10:10:54 -0400


--------_3ED8A3BF557D04F17FC0_MULTIPART_MIXED_
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit

Although the keyboard (keyboard.c) and terminal (vt.c) both have "UTF-8
modes", there are still a couple of short-comings at the kernel level.

* compose tables use 8-bit characters,
* selection doesn't copy/paste UTF-8.

(OK, so there are many more, but these two are the ones that make
it inconvenient to even use extended Latin characters.)

Here is a patch against 2.5.70 that fixes selection. It uses the
existing Unicode font information (ushorts) to create an inverse mapping.

Chris

--------_3ED8A3BF557D04F17FC0_MULTIPART_MIXED_
Content-Type: application/octet-stream; name="selection.patch"
Content-Disposition: attachment;
filename="selection.patch"
Content-Transfer-Encoding: base64

LS0tIGEvaW5jbHVkZS9saW51eC9jb25zb2xlbWFwLmgJMjAwMy0wNS0wNCAxOTo1Mzo1Ni4wMDAw
MDAwMDAgLTA0MDAKKysrIGIvaW5jbHVkZS9saW51eC9jb25zb2xlbWFwLmgJMjAwMy0wNS0yNiAx
NToxNzo0NS4wMDAwMDAwMDAgLTA0MDAKQEAgLTEwLDYgKzEwLDYgQEAKIAogc3RydWN0IHZjX2Rh
dGE7CiAKLWV4dGVybiB1bnNpZ25lZCBjaGFyIGludmVyc2VfdHJhbnNsYXRlKHN0cnVjdCB2Y19k
YXRhICpjb25wLCBpbnQgZ2x5cGgpOworZXh0ZXJuIHUxNiBpbnZlcnNlX3RyYW5zbGF0ZShzdHJ1
Y3QgdmNfZGF0YSAqY29ucCwgaW50IGdseXBoLCBpbnQgdXNlX3VuaWNvZGUpOwogZXh0ZXJuIHVu
c2lnbmVkIHNob3J0ICpzZXRfdHJhbnNsYXRlKGludCBtLGludCBjdXJyY29ucyk7CiBleHRlcm4g
aW50IGNvbnZfdW5pX3RvX3BjKHN0cnVjdCB2Y19kYXRhICpjb25wLCBsb25nIHVjcyk7Ci0tLSBh
L2RyaXZlcnMvY2hhci9jb25zb2xlbWFwLmMJMjAwMy0wNS0wNCAxOTo1MzozNi4wMDAwMDAwMDAg
LTA0MDAKKysrIGIvZHJpdmVycy9jaGFyL2NvbnNvbGVtYXAuYwkyMDAzLTA1LTI2IDE1OjAxOjU4
LjAwMDAwMDAwMCAtMDQwMApAQCAtMTc4LDYgKzE3OCw3IEBACiAJdW5zaWduZWQgbG9uZwlyZWZj
b3VudDsKIAl1bnNpZ25lZCBsb25nCXN1bTsKIAl1bnNpZ25lZCBjaGFyCSppbnZlcnNlX3RyYW5z
bGF0aW9uc1s0XTsKKwl1MTYJCSppbnZlcnNlX3RyYW5zX3VuaWNvZGU7CiAJaW50CQlyZWFkb25s
eTsKIH07CiAKQEAgLTIwOCw2ICsyMDksMzkgQEAKIAl9CiB9CiAKK3N0YXRpYyB2b2lkIHNldF9p
bnZlcnNlX3RyYW5zX3VuaWNvZGUoc3RydWN0IHZjX2RhdGEgKmNvbnAsIHN0cnVjdCB1bmlfcGFn
ZWRpciAqcCkKK3sKKwlpbnQgaSwgaiwgaywgZ2x5cGg7CisJdTE2ICoqcDEsICpwMjsKKwl1MTYg
KnE7CisJCisJaWYgKCFwKSByZXR1cm47CisJcSA9IHAtPmludmVyc2VfdHJhbnNfdW5pY29kZTsK
KworCWlmICghcSkgeworCQlxID0gcC0+aW52ZXJzZV90cmFuc191bmljb2RlID0gKHUxNiAqKSAK
KwkJCWttYWxsb2MoTUFYX0dMWVBIICogc2l6ZW9mKHUxNiksIEdGUF9LRVJORUwpOworCQlpZiAo
IXEpIHJldHVybjsKKwl9CisJbWVtc2V0KHEsIDAsIE1BWF9HTFlQSCAqIHNpemVvZih1MTYpKTsK
KworCWZvciAoaSA9IDA7IGkgPCAzMjsgaSsrKSB7CisJCXAxID0gcC0+dW5pX3BnZGlyW2ldOwor
CQlpZiAoIXAxKQorCQkJY29udGludWU7CisJCWZvciAoaiA9IDA7IGogPCAzMjsgaisrKSB7CisJ
CQlwMiA9IHAxW2pdOworCQkJaWYgKCFwMikKKwkJCQljb250aW51ZTsKKwkJCWZvciAoayA9IDA7
IGsgPCA2NDsgaysrKSB7CisJCQkJZ2x5cGggPSBwMltrXTsKKwkJCQlpZiAoZ2x5cGggPj0gMCAm
JiBnbHlwaCA8IE1BWF9HTFlQSCAmJiBxW2dseXBoXSA8IDMyKQorCQkgIAkJCXFbZ2x5cGhdID0g
KGkgPDwgMTEpICsgKGogPDwgNikgKyBrOworCQkJfQorCQl9CisJfQorfQorCiB1bnNpZ25lZCBz
aG9ydCAqc2V0X3RyYW5zbGF0ZShpbnQgbSxpbnQgY3VycmNvbnMpCiB7CiAJaW52X3RyYW5zbGF0
ZVtjdXJyY29uc10gPSBtOwpAQCAtMjE4LDE5ICsyNTIsMjcgQEAKICAqIEludmVyc2UgdHJhbnNs
YXRpb24gaXMgaW1wb3NzaWJsZSBmb3Igc2V2ZXJhbCByZWFzb25zOgogICogMS4gVGhlIGZvbnQ8
LT5jaGFyYWN0ZXIgbWFwcyBhcmUgbm90IDEtMS4KICAqIDIuIFRoZSB0ZXh0IG1heSBoYXZlIGJl
ZW4gd3JpdHRlbiB3aGlsZSBhIGRpZmZlcmVudCB0cmFuc2xhdGlvbiBtYXAKLSAqICAgIHdhcyBh
Y3RpdmUsIG9yIHVzaW5nIFVuaWNvZGUuCisgKiAgICB3YXMgYWN0aXZlLgogICogU3RpbGwsIGl0
IGlzIG5vdyBwb3NzaWJsZSB0byBhIGNlcnRhaW4gZXh0ZW50IHRvIGN1dCBhbmQgcGFzdGUgbm9u
LUFTQ0lJLgogICovCi11bnNpZ25lZCBjaGFyIGludmVyc2VfdHJhbnNsYXRlKHN0cnVjdCB2Y19k
YXRhICpjb25wLCBpbnQgZ2x5cGgpCit1MTYgaW52ZXJzZV90cmFuc2xhdGUoc3RydWN0IHZjX2Rh
dGEgKmNvbnAsIGludCBnbHlwaCwgaW50IHVzZV91bmljb2RlKQogewogCXN0cnVjdCB1bmlfcGFn
ZWRpciAqcDsKIAlpZiAoZ2x5cGggPCAwIHx8IGdseXBoID49IE1BWF9HTFlQSCkKIAkJcmV0dXJu
IDA7Ci0JZWxzZSBpZiAoIShwID0gKHN0cnVjdCB1bmlfcGFnZWRpciAqKSpjb25wLT52Y191bmlf
cGFnZWRpcl9sb2MpIHx8Ci0JCSAhcC0+aW52ZXJzZV90cmFuc2xhdGlvbnNbaW52X3RyYW5zbGF0
ZVtjb25wLT52Y19udW1dXSkKKwllbHNlIGlmICghKHAgPSAoc3RydWN0IHVuaV9wYWdlZGlyICop
KmNvbnAtPnZjX3VuaV9wYWdlZGlyX2xvYykpCiAJCXJldHVybiBnbHlwaDsKLQllbHNlCi0JCXJl
dHVybiBwLT5pbnZlcnNlX3RyYW5zbGF0aW9uc1tpbnZfdHJhbnNsYXRlW2NvbnAtPnZjX251bV1d
W2dseXBoXTsKKwllbHNlIGlmICh1c2VfdW5pY29kZSkgeworCQlpZiAoIXAtPmludmVyc2VfdHJh
bnNfdW5pY29kZSkKKwkJCXJldHVybiBnbHlwaDsKKwkJZWxzZQorCQkJcmV0dXJuIHAtPmludmVy
c2VfdHJhbnNfdW5pY29kZVtnbHlwaF07CisJfSBlbHNlIHsKKwkJaWYgKCFwLT5pbnZlcnNlX3Ry
YW5zbGF0aW9uc1tpbnZfdHJhbnNsYXRlW2NvbnAtPnZjX251bV1dKQorCQkJcmV0dXJuIGdseXBo
OworCQllbHNlCisJCQlyZXR1cm4gcC0+aW52ZXJzZV90cmFuc2xhdGlvbnNbaW52X3RyYW5zbGF0
ZVtjb25wLT52Y19udW1dXVtnbHlwaF07CisJfQogfQogCiBzdGF0aWMgdm9pZCB1cGRhdGVfdXNl
cl9tYXBzKHZvaWQpCkBAIC0zNjIsNiArNDA0LDEwIEBACiAJCQlrZnJlZShwLT5pbnZlcnNlX3Ry
YW5zbGF0aW9uc1tpXSk7CiAJCQlwLT5pbnZlcnNlX3RyYW5zbGF0aW9uc1tpXSA9IE5VTEw7CiAJ
CX0KKwlpZiAocC0+aW52ZXJzZV90cmFuc191bmljb2RlKSB7CisJCWtmcmVlKHAtPmludmVyc2Vf
dHJhbnNfdW5pY29kZSk7CisJCXAtPmludmVyc2VfdHJhbnNfdW5pY29kZSA9IE5VTEw7CisJfQog
fQogCiB2b2lkIGNvbl9mcmVlX3VuaW1hcChpbnQgY29uKQpAQCAtNTIyLDYgKzU2OCw3IEBACiAK
IAlmb3IgKGkgPSAwOyBpIDw9IDM7IGkrKykKIAkJc2V0X2ludmVyc2VfdHJhbnNsKGNvbnAsIHAs
IGkpOyAvKiBVcGRhdGUgYWxsIGludmVyc2UgdHJhbnNsYXRpb25zICovCisJc2V0X2ludmVyc2Vf
dHJhbnNfdW5pY29kZShjb25wLCBwKTsKICAgCiAJcmV0dXJuIGVycjsKIH0KQEAgLTU3NCw2ICs2
MjEsNyBAQAogCiAJZm9yIChpID0gMDsgaSA8PSAzOyBpKyspCiAJCXNldF9pbnZlcnNlX3RyYW5z
bChjb25wLCBwLCBpKTsJLyogVXBkYXRlIGFsbCBpbnZlcnNlIHRyYW5zbGF0aW9ucyAqLworCXNl
dF9pbnZlcnNlX3RyYW5zX3VuaWNvZGUoY29ucCwgcCk7CiAJZGZsdCA9IHA7CiAJcmV0dXJuIGVy
cjsKIH0KLS0tIGEvZHJpdmVycy9jaGFyL3NlbGVjdGlvbi5jCTIwMDMtMDUtMDQgMTk6NTM6NDIu
MDAwMDAwMDAwIC0wNDAwCisrKyBiL2RyaXZlcnMvY2hhci9zZWxlY3Rpb24uYwkyMDAzLTA1LTI2
IDE1OjI2OjA3LjAwMDAwMDAwMCAtMDQwMApAQCAtMjAsNiArMjAsNyBAQAogCiAjaW5jbHVkZSA8
YXNtL3VhY2Nlc3MuaD4KIAorI2luY2x1ZGUgPGxpbnV4L2tiZF9rZXJuLmg+CiAjaW5jbHVkZSA8
bGludXgvdnRfa2Vybi5oPgogI2luY2x1ZGUgPGxpbnV4L2NvbnNvbGVtYXAuaD4KICNpbmNsdWRl
IDxsaW51eC9zZWxlY3Rpb24uaD4KQEAgLTM2LDYgKzM3LDcgQEAKIC8qIFZhcmlhYmxlcyBmb3Ig
c2VsZWN0aW9uIGNvbnRyb2wuICovCiAvKiBVc2UgYSBkeW5hbWljIGJ1ZmZlciwgaW5zdGVhZCBv
ZiBzdGF0aWMgKERlYyAxOTk0KSAqLwogICAgICAgIGludCBzZWxfY29uczsJCS8qIG11c3Qgbm90
IGJlIGRpc2FsbG9jYXRlZCAqLworc3RhdGljIGludCB1c2VfdW5pY29kZTsKIHN0YXRpYyB2b2xh
dGlsZSBpbnQgc2VsX3N0YXJ0ID0gLTE7IAkvKiBjbGVhcmVkIGJ5IGNsZWFyX3NlbGVjdGlvbiAq
Lwogc3RhdGljIGludCBzZWxfZW5kOwogc3RhdGljIGludCBzZWxfYnVmZmVyX2x0aDsKQEAgLTU2
LDEwICs1OCwxMCBAQAogCWNvbXBsZW1lbnRfcG9zKHNlbF9jb25zLCB3aGVyZSk7CiB9CiAKLXN0
YXRpYyB1bnNpZ25lZCBjaGFyCitzdGF0aWMgdTE2CiBzZWxfcG9zKGludCBuKQogewotCXJldHVy
biBpbnZlcnNlX3RyYW5zbGF0ZSh2Y19jb25zW3NlbF9jb25zXS5kLCBzY3JlZW5fZ2x5cGgoc2Vs
X2NvbnMsIG4pKTsKKwlyZXR1cm4gaW52ZXJzZV90cmFuc2xhdGUodmNfY29uc1tzZWxfY29uc10u
ZCwgc2NyZWVuX2dseXBoKHNlbF9jb25zLCBuKSwgdXNlX3VuaWNvZGUpOwogfQogCiAvKiByZW1v
dmUgdGhlIGN1cnJlbnQgc2VsZWN0aW9uIGhpZ2hsaWdodCwgaWYgYW55LApAQCAtODgsOCArOTAs
OCBAQAogICAweEZGN0ZGRkZGICAvKiBsYXRpbi0xIGFjY2VudGVkIGxldHRlcnMsIG5vdCBkaXZp
c2lvbiBzaWduICovCiB9OwogCi1zdGF0aWMgaW5saW5lIGludCBpbndvcmQoY29uc3QgdW5zaWdu
ZWQgY2hhciBjKSB7Ci0JcmV0dXJuICggaW53b3JkTHV0W2M+PjVdID4+IChjICYgMHgxRikgKSAm
IDE7CitzdGF0aWMgaW5saW5lIGludCBpbndvcmQoY29uc3QgdTE2IGMpIHsKKwlyZXR1cm4gYyA+
IDB4ZmYgfHwgKCggaW53b3JkTHV0W2M+PjVdID4+IChjICYgMHgxRikgKSAmIDEpOwogfQogCiAv
KiBzZXQgaW53b3JkTHV0IGNvbnRlbnRzLiBJbnZva2VkIGJ5IGlvY3RsKCkuICovCkBAIC0xMTAs
MTMgKzExMiwzNiBAQAogCXJldHVybiAodiA+IHUpID8gdSA6IHY7CiB9CiAKKy8qIHN0b3JlcyB0
aGUgY2hhciBpbiBVVEY4IGFuZCByZXR1cm5zIHRoZSBudW1iZXIgb2YgYnl0ZXMgdXNlZCAoMS0z
KSAqLworaW50IHN0b3JlX3V0ZjgodTE2IGMsIGNoYXIgKnApIAoreworCWlmIChjIDwgMHg4MCkg
eworCQkvKiAgMCoqKioqKiogKi8KKwkJcFswXSA9IGM7CisJCXJldHVybiAxOworCX0gZWxzZSBp
ZiAoYyA8IDB4ODAwKSB7CisJCS8qIDExMCoqKioqIDEwKioqKioqICovCisJCXBbMF0gPSAweGMw
IHwgKGMgPj4gNik7CisJCXBbMV0gPSAweDgwIHwgKGMgJiAweDNmKTsKKwkJcmV0dXJuIDI7Cisg
ICAgCX0gZWxzZSB7CisJCS8qIDExMTAqKioqIDEwKioqKioqIDEwKioqKioqICovCisJCXBbMF0g
PSAweGUwIHwgKGMgPj4gMTIpOworCQlwWzFdID0gMHg4MCB8ICgoYyA+PiA2KSAmIDB4M2YpOwor
CQlwWzJdID0gMHg4MCB8IChjICYgMHgzZik7CisJCXJldHVybiAzOworICAgIAl9Cit9CisKIC8q
IHNldCB0aGUgY3VycmVudCBzZWxlY3Rpb24uIEludm9rZWQgYnkgaW9jdGwoKSBvciBieSBrZXJu
ZWwgY29kZS4gKi8KIGludCBzZXRfc2VsZWN0aW9uKGNvbnN0IHVuc2lnbmVkIGxvbmcgYXJnLCBz
dHJ1Y3QgdHR5X3N0cnVjdCAqdHR5LCBpbnQgdXNlcikKIHsKIAlpbnQgc2VsX21vZGUsIG5ld19z
ZWxfc3RhcnQsIG5ld19zZWxfZW5kLCBzcGM7CiAJY2hhciAqYnAsICpvYnA7Ci0JaW50IGksIHBz
LCBwZTsKKwlpbnQgaSwgcHMsIHBlLCBtdWx0aXBsaWVyOworCXUxNiBjOwogCXVuc2lnbmVkIGlu
dCBjdXJyY29ucyA9IGZnX2NvbnNvbGU7CisJc3RydWN0IGtiZF9zdHJ1Y3QgKmtiZCA9IGtiZF90
YWJsZSArIGZnX2NvbnNvbGU7CiAKIAl1bmJsYW5rX3NjcmVlbigpOwogCXBva2VfYmxhbmtlZF9j
b25zb2xlKCk7CkBAIC0xNzAsNiArMTk1LDcgQEAKIAkJY2xlYXJfc2VsZWN0aW9uKCk7CiAJCXNl
bF9jb25zID0gZmdfY29uc29sZTsKIAl9CisJdXNlX3VuaWNvZGUgPSBrYmQgJiYga2JkLT5rYmRt
b2RlID09IFZDX1VOSUNPREU7CiAKIAlzd2l0Y2ggKHNlbF9tb2RlKQogCXsKQEAgLTI1Miw3ICsy
NzgsOCBAQAogCXNlbF9lbmQgPSBuZXdfc2VsX2VuZDsKIAogCS8qIEFsbG9jYXRlIGEgbmV3IGJ1
ZmZlciBiZWZvcmUgZnJlZWluZyB0aGUgb2xkIG9uZSAuLi4gKi8KLQlicCA9IGttYWxsb2MoKHNl
bF9lbmQtc2VsX3N0YXJ0KS8yKzEsIEdGUF9LRVJORUwpOworCW11bHRpcGxpZXIgPSB1c2VfdW5p
Y29kZSA/IDMgOiAxOyAgLyogY2hhcnMgY2FuIHRha2UgdXAgdG8gMyBieXRlcyAqLworCWJwID0g
a21hbGxvYygoc2VsX2VuZC1zZWxfc3RhcnQpLzIqbXVsdGlwbGllcisxLCBHRlBfS0VSTkVMKTsK
IAlpZiAoIWJwKSB7CiAJCXByaW50ayhLRVJOX1dBUk5JTkcgInNlbGVjdGlvbjoga21hbGxvYygp
IGZhaWxlZFxuIik7CiAJCWNsZWFyX3NlbGVjdGlvbigpOwpAQCAtMjY0LDggKzI5MSwxMiBAQAog
CiAJb2JwID0gYnA7CiAJZm9yIChpID0gc2VsX3N0YXJ0OyBpIDw9IHNlbF9lbmQ7IGkgKz0gMikg
ewotCQkqYnAgPSBzZWxfcG9zKGkpOwotCQlpZiAoIWlzc3BhY2UoKmJwKyspKQorCQljID0gc2Vs
X3BvcyhpKTsKKwkJaWYgKHVzZV91bmljb2RlKQorCQkJYnAgKz0gc3RvcmVfdXRmOChjLCBicCk7
CisJCWVsc2UKKwkJCSpicCsrID0gYzsKKwkJaWYgKCFpc3NwYWNlKGMpKQogCQkJb2JwID0gYnA7
CiAJCWlmICghICgoaSArIDIpICUgdmlkZW9fc2l6ZV9yb3cpKSB7CiAJCQkvKiBzdHJpcCB0cmFp
bGluZyBibGFua3MgZnJvbSBsaW5lIGFuZCBhZGQgbmV3bGluZSwK

--------_3ED8A3BF557D04F17FC0_MULTIPART_MIXED_--

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/