Module string/unicode
Unicode case conversion — locale-independent, table-driven.
to_lowercase_bytes / to_uppercase_bytes apply the Unicode 15.1.0
case mappings: the simple (one-to-one) mappings from UnicodeData.txt for
every script, plus the one-to-many SpecialCasing expansions (ß → SS,
fi → FI, İ → i̇, …). They never consult the C locale —
towlower/towupper only map ASCII under the default "C" locale,
which is what this module used to call
(issues/fixed/unicode-case-mapping-was-locale-dependent.md).
Context-sensitive rules (Greek final sigma, Lithuanian / Turkish dotted
i) are NOT applied; the mapping is per code point, like Go's
unicode.ToUpper and strings.ToUpper.
This is a LEAF module (byte buffers in, byte buffers out — no String
import) so that std/string/string.yo can import IT and route
String.to_lowercase/to_uppercase through these tables without a
dependency cycle. The String-level entry points are those methods.
Example
{ to_lowercase_bytes } :: import "std/string/unicode";
lower := `HELLO WÖRLD`.to_lowercase(); // "hello wörld" — String method
Stability
unstable — the four exported entry points (to_lowercase_bytes,
to_uppercase_bytes, to_lower_code_point, to_upper_code_point) are the
right shape and are unlikely to move, but the tables behind them are
GENERATED against one Unicode version (15.1.0, via Python unicodedata;
generator issues/repros/gen_unicode_case_table.py) and there is no
regeneration step in the build, so a Unicode upgrade is a manual
re-derivation of five parallel arrays. The other open question is scope:
the audit wants Unicode-aware rune classifiers
(plans/STD_API_STABILIZATION.md §4, "Text"), which means this module
grows category tables beside the case tables and may need a different
lookup shape than the alternating-range trick the case table uses.
Freezing follows that growth and a regeneration story, not a release count.
Functions
Lowercase UTF-8 bytes using the Unicode case mapping (all scripts, plus
the İ → i̇ expansion). Locale-independent. String.to_lowercase calls
this; the input must be valid UTF-8 (a String always is).
Parameters
| Name | Type | Notes |
|---|---|---|
input | ArrayList(u8) |
Returns: ArrayList(u8)
Uppercase UTF-8 bytes using the Unicode case mapping (all scripts, plus
the one-to-many expansions such as ß → SS and fi → FI).
Locale-independent. String.to_uppercase calls this; the input must be
valid UTF-8 (a String always is).
Parameters
| Name | Type | Notes |
|---|---|---|
input | ArrayList(u8) |
Returns: ArrayList(u8)
The simple (one-to-one) uppercase mapping of a code point, or the code point itself when it has none.
Parameters
| Name | Type | Notes |
|---|---|---|
cp | i32 |
Returns: i32
The simple (one-to-one) lowercase mapping of a code point, or the code point itself when it has none.
Parameters
| Name | Type | Notes |
|---|---|---|
cp | i32 |
Returns: i32