unicode

Branch: main

Switch branches or tags

Branch: main
Go to file t

413 files indexed

History
Code

casefold/README.md

unicode-casefold

Unicode 18.0.0 full case folding and caseless matching for MoonBit. The package depends on unicode-normalization for normalized folding keys.

Installation

moon add moonbit-community/unicode-casefold
import {
  "moonbit-community/unicode-casefold" @casefold,
}

Usage

let folded = @casefold.case_fold("Straße") // "strasse"
let same_case = @casefold.caseless_match("Straße", "STRASSE") // true
let same_text = @casefold.canonical_caseless_match("Å", "a\u{030A}") // true
let search_key = @casefold.canonical_case_fold("Å") // "a\u{030A}"
let folded_id = @casefold.nfkc_casefold("A\u{00AD}") // "a"
let same_id = @casefold.identifier_caseless_match("A\u{00AD}", "a") // true

These operations follow Unicode §3.13 and are locale-independent. Default full folding uses CaseFolding.txt statuses C and F, with expansions such as ß → ss; it does not normalize text or apply Turkic tailoring. I folds to i, while İ folds to i plus U+0307. Cherokee folds to uppercase, so a folded string is not necessarily lowercase.

For canonically equivalent text, canonical_case_fold(s) computes NFD(case_fold(NFD(s))) (D145). Store and compare these keys, or search folded text with a folded query. Folding can change text length, so result offsets do not directly identify positions in the original text.

nfkc_casefold(s) applies the official NFKC_CF mappings and then NFC (R5). This merges compatibility differences and removes default-ignorable characters. identifier_caseless_match(a, b) first applies NFD to both inputs before NFKC_Casefold (D147), including the Greek iota-subscript edge cases. It tests equality; identifier validation and confusable detection are separate operations.

API

API Description
case_fold(StringView) -> String Default full case folding (R4).
caseless_match(StringView, StringView) -> Bool Default caseless equality, without normalization (D144).
canonical_case_fold(StringView) -> String Canonical caseless comparison/search key (D145).
canonical_caseless_match(StringView, StringView) -> Bool Caseless equality under canonical equivalence (D145).
nfkc_casefold(StringView) -> String Official NFKC_CF mapping followed by NFC (R5).
identifier_caseless_match(StringView, StringView) -> Bool Identifier caseless equality after initial NFD (D147).

All text inputs accept StringView, including slices of larger strings. Existing calls with String arguments work through automatic view conversion. Text-producing operations still return String.

1# unicode-casefold23Unicode 18.0.0 full case folding and caseless matching for MoonBit.4The package depends on `unicode-normalization` for normalized folding keys.56## Installation78```bash9moon add moonbit-community/unicode-casefold10```1112```moonbit13import {14  "moonbit-community/unicode-casefold" @casefold,15}16```1718## Usage1920```moonbit21let folded = @casefold.case_fold("Straße") // "strasse"22let same_case = @casefold.caseless_match("Straße", "STRASSE") // true23let same_text = @casefold.canonical_caseless_match("Å", "a\u{030A}") // true24let search_key = @casefold.canonical_case_fold("Å") // "a\u{030A}"25let folded_id = @casefold.nfkc_casefold("A\u{00AD}") // "a"26let same_id = @casefold.identifier_caseless_match("A\u{00AD}", "a") // true27```2829These operations follow [Unicode §3.13](https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-3/)30and are locale-independent. Default full folding uses `CaseFolding.txt` statuses31`C` and `F`, with expansions such as `ß → ss`; it does not normalize text or32apply Turkic tailoring. `I` folds to `i`, while `İ` folds to `i` plus U+0307.33Cherokee folds to uppercase, so a folded string is not necessarily lowercase.3435For canonically equivalent text, `canonical_case_fold(s)` computes36`NFD(case_fold(NFD(s)))` (D145). Store and compare these keys, or search folded37text with a folded query. Folding can change text length, so result offsets do38not directly identify positions in the original text.3940`nfkc_casefold(s)` applies the official `NFKC_CF` mappings and then NFC (R5).41This merges compatibility differences and removes default-ignorable characters.42`identifier_caseless_match(a, b)` first applies NFD to both inputs before43NFKC_Casefold (D147), including the Greek iota-subscript edge cases. It tests44equality; identifier validation and confusable detection are separate operations.4546## API4748| API | Description |49| --- | --- |50| `case_fold(StringView) -> String` | Default full case folding (R4). |51| `caseless_match(StringView, StringView) -> Bool` | Default caseless equality, without normalization (D144). |52| `canonical_case_fold(StringView) -> String` | Canonical caseless comparison/search key (D145). |53| `canonical_caseless_match(StringView, StringView) -> Bool` | Caseless equality under canonical equivalence (D145). |54| `nfkc_casefold(StringView) -> String` | Official NFKC_CF mapping followed by NFC (R5). |55| `identifier_caseless_match(StringView, StringView) -> Bool` | Identifier caseless equality after initial NFD (D147). |5657All text inputs accept `StringView`, including slices of larger strings.58Existing calls with `String` arguments work through automatic view conversion.59Text-producing operations still return `String`.60