Module string/unicode

string/unicode
Stability: unstable — the four exported entry points (`to_lowercase_bytes`, `to_uppercase_bytes`, `to_lower_code_point`, `to_upper_code_point`) are the right shape and are unlikely to move, but the tables behind them are GENERATED against one Unicode version (15.1.0, via Python `unicodedata`; generator `issues/repros/gen_unicode_case_table.py`) and there is no regeneration step in the build, so a Unicode upgrade is a manual re-derivation of five parallel arrays. The other open question is scope: the audit wants Unicode-aware `rune` classifiers (`plans/STD_API_STABILIZATION.md` §4, "Text"), which means this module grows category tables beside the case tables and may need a different lookup shape than the alternating-range trick the case table uses. Freezing follows that growth and a regeneration story, not a release count. — stable modules only change additively; this one may still change.

Unicode case conversion — locale-independent, table-driven.

to_lowercase_bytes / to_uppercase_bytes apply the Unicode 15.1.0 case mappings: the simple (one-to-one) mappings from UnicodeData.txt for every script, plus the one-to-many SpecialCasing expansions (ßSS, FI, İ, …). They never consult the C locale — towlower/towupper only map ASCII under the default "C" locale, which is what this module used to call (issues/fixed/unicode-case-mapping-was-locale-dependent.md).

Context-sensitive rules (Greek final sigma, Lithuanian / Turkish dotted i) are NOT applied; the mapping is per code point, like Go's unicode.ToUpper and strings.ToUpper.

This is a LEAF module (byte buffers in, byte buffers out — no String import) so that std/string/string.yo can import IT and route String.to_lowercase/to_uppercase through these tables without a dependency cycle. The String-level entry points are those methods.

Example

{ to_lowercase_bytes } :: import "std/string/unicode";

lower := `HELLO WÖRLD`.to_lowercase();  // "hello wörld" — String method

Stability

unstable — the four exported entry points (to_lowercase_bytes, to_uppercase_bytes, to_lower_code_point, to_upper_code_point) are the right shape and are unlikely to move, but the tables behind them are GENERATED against one Unicode version (15.1.0, via Python unicodedata; generator issues/repros/gen_unicode_case_table.py) and there is no regeneration step in the build, so a Unicode upgrade is a manual re-derivation of five parallel arrays. The other open question is scope: the audit wants Unicode-aware rune classifiers (plans/STD_API_STABILIZATION.md §4, "Text"), which means this module grows category tables beside the case tables and may need a different lookup shape than the alternating-range trick the case table uses. Freezing follows that growth and a regeneration story, not a release count.

Functions

fn(input : ArrayList(u8)) -> ArrayList(u8)

Lowercase UTF-8 bytes using the Unicode case mapping (all scripts, plus the İ expansion). Locale-independent. String.to_lowercase calls this; the input must be valid UTF-8 (a String always is).

Parameters

NameTypeNotes
inputArrayList(u8)

Returns: ArrayList(u8)

fn(input : ArrayList(u8)) -> ArrayList(u8)

Uppercase UTF-8 bytes using the Unicode case mapping (all scripts, plus the one-to-many expansions such as ßSS and FI). Locale-independent. String.to_uppercase calls this; the input must be valid UTF-8 (a String always is).

Parameters

NameTypeNotes
inputArrayList(u8)

Returns: ArrayList(u8)

fn(cp : i32) -> i32

The simple (one-to-one) uppercase mapping of a code point, or the code point itself when it has none.

Parameters

NameTypeNotes
cpi32

Returns: i32

fn(cp : i32) -> i32

The simple (one-to-one) lowercase mapping of a code point, or the code point itself when it has none.

Parameters

NameTypeNotes
cpi32

Returns: i32