unicode
Unicode Module Split
Status: Validated
Outcome
Split the published moonbit-community/unicode module into independently consumable
modules so users only download the Unicode functionality they use:
moonbit-community/ucdmoonbit-community/normalizationmoonbit-community/punycodemoonbit-community/bidimoonbit-community/idna
Keep moonbit-community/unicode as a compatibility umbrella whose existing package
paths forward to the new modules.
Boundaries
- Each feature module owns its public types and implementation data.
moonbit-community/ucd/dataexposes the low-level table lookups required by normalization and IDNA; user-facing case and category APIs remain atmoonbit-community/ucd.BidiClassis owned by the publicmoonbit-community/bidipackage rather than an internal package.moonbit-community/idnadepends on UCD, normalization, Punycode, and Bidi.- No persistence, wire-format, security-policy, or third-party dependency changes are included.
Tests and generation
Unit and fuzz tests stay with their owning feature modules. Large generated
Unicode, IDNA, and Bidi conformance fixtures move to a workspace-only
moonbit-community/unicode-conformance module. They remain part of local and CI test
runs but are excluded from published .mooncakes archives.
The existing moonbit-community/unicode-tools module remains workspace-only and is
updated to emit data, interfaces, and conformance fixtures at the new paths.
Compatibility
The umbrella module keeps the existing public package paths:
moonbit-community/unicodemoonbit-community/unicode/normalizationmoonbit-community/unicode/punycodemoonbit-community/unicode/bidimoonbit-community/unicode/idna
These packages are thin facades over the new modules. New consumers should depend directly on the feature module they need.
Validation
- Run
moon fmtandmoon infofor the workspace. - Run
moon checkandmoon testfor every feature module and for the whole workspace. - Regenerate all Unicode data and conformance fixtures, then repeat checks.
- Run
moon package --listin every published module. - Record archive sizes and verify conformance fixtures are absent.
Validation evidence
moon run --target native tools/gen allcompleted with cached Unicode 16.0.0 source data and the default Bidi part size of 500.moon fmt,moon info, andmoon checkcompleted without warnings.moon testpassed all 8422 tests, including the workspace-only generated conformance suites and legacy-package compatibility tests.- The five compatibility
.mbtifiles are identical to the interfaces at commitf3e356f1. moon package --listand zip integrity checks passed for all six published modules. No archive contains a conformance path.- Archive sizes are 16 KB (
unicode), 137 KB (ucd), 15 KB (normalization), 10 KB (punycode), 27 KB (bidi), and 107 KB (idna).
Rollback
The split is delivered in coherent commits. Before publication, rollback is a normal Git revert to the original single-module layout; no external state or stored data migration is involved.
1# Unicode Module Split23Status: Validated45## Outcome67Split the published `moonbit-community/unicode` module into independently consumable8modules so users only download the Unicode functionality they use:910- `moonbit-community/ucd`11- `moonbit-community/normalization`12- `moonbit-community/punycode`13- `moonbit-community/bidi`14- `moonbit-community/idna`1516Keep `moonbit-community/unicode` as a compatibility umbrella whose existing package17paths forward to the new modules.1819## Boundaries2021- Each feature module owns its public types and implementation data.22- `moonbit-community/ucd/data` exposes the low-level table lookups required by23 normalization and IDNA; user-facing case and category APIs remain at24 `moonbit-community/ucd`.25- `BidiClass` is owned by the public `moonbit-community/bidi` package rather than an26 internal package.27- `moonbit-community/idna` depends on UCD, normalization, Punycode, and Bidi.28- No persistence, wire-format, security-policy, or third-party dependency29 changes are included.3031## Tests and generation3233Unit and fuzz tests stay with their owning feature modules. Large generated34Unicode, IDNA, and Bidi conformance fixtures move to a workspace-only35`moonbit-community/unicode-conformance` module. They remain part of local and CI test36runs but are excluded from published `.mooncakes` archives.3738The existing `moonbit-community/unicode-tools` module remains workspace-only and is39updated to emit data, interfaces, and conformance fixtures at the new paths.4041## Compatibility4243The umbrella module keeps the existing public package paths:4445- `moonbit-community/unicode`46- `moonbit-community/unicode/normalization`47- `moonbit-community/unicode/punycode`48- `moonbit-community/unicode/bidi`49- `moonbit-community/unicode/idna`5051These packages are thin facades over the new modules. New consumers should52depend directly on the feature module they need.5354## Validation55561. Run `moon fmt` and `moon info` for the workspace.572. Run `moon check` and `moon test` for every feature module and for the whole58 workspace.593. Regenerate all Unicode data and conformance fixtures, then repeat checks.604. Run `moon package --list` in every published module.615. Record archive sizes and verify conformance fixtures are absent.6263## Validation evidence6465- `moon run --target native tools/gen all` completed with cached Unicode 16.0.066 source data and the default Bidi part size of 500.67- `moon fmt`, `moon info`, and `moon check` completed without warnings.68- `moon test` passed all 8422 tests, including the workspace-only generated69 conformance suites and legacy-package compatibility tests.70- The five compatibility `.mbti` files are identical to the interfaces at71 commit `f3e356f1`.72- `moon package --list` and zip integrity checks passed for all six published73 modules. No archive contains a conformance path.74- Archive sizes are 16 KB (`unicode`), 137 KB (`ucd`), 15 KB75 (`normalization`), 10 KB (`punycode`), 27 KB (`bidi`), and 107 KB (`idna`).7677## Rollback7879The split is delivered in coherent commits. Before publication, rollback is a80normal Git revert to the original single-module layout; no external state or81stored data migration is involved.82# Unicode Module Split Status: Validated ## Outcome Split the published `moonbit-community/unicode` module into independently consumable modules so users only download the Unicode functionality they use: - `moonbit-community/ucd` - `moonbit-community/normalization` - `moonbit-community/punycode` - `moonbit-community/bidi` - `moonbit-community/idna` Keep `moonbit-community/unicode` as a compatibility umbrella whose existing package paths forward to the new modules. ## Boundaries - Each feature module owns its public types and implementation data. - `moonbit-community/ucd/data` exposes the low-level table lookups required by normalization and IDNA; user-facing case and category APIs remain at `moonbit-community/ucd`. - `BidiClass` is owned by the public `moonbit-community/bidi` package rather than an internal package. - `moonbit-community/idna` depends on UCD, normalization, Punycode, and Bidi. - No persistence, wire-format, security-policy, or third-party dependency changes are included. ## Tests and generation Unit and fuzz tests stay with their owning feature modules. Large generated Unicode, IDNA, and Bidi conformance fixtures move to a workspace-only `moonbit-community/unicode-conformance` module. They remain part of local and CI test runs but are excluded from published `.mooncakes` archives. The existing `moonbit-community/unicode-tools` module remains workspace-only and is updated to emit data, interfaces, and conformance fixtures at the new paths. ## Compatibility The umbrella module keeps the existing public package paths: - `moonbit-community/unicode` - `moonbit-community/unicode/normalization` - `moonbit-community/unicode/punycode` - `moonbit-community/unicode/bidi` - `moonbit-community/unicode/idna` These packages are thin facades over the new modules. New consumers should depend directly on the feature module they need. ## Validation 1. Run `moon fmt` and `moon info` for the workspace. 2. Run `moon check` and `moon test` for every feature module and for the whole workspace. 3. Regenerate all Unicode data and conformance fixtures, then repeat checks. 4. Run `moon package --list` in every published module. 5. Record archive sizes and verify conformance fixtures are absent. ## Validation evidence - `moon run --target native tools/gen all` completed with cached Unicode 16.0.0 source data and the default Bidi part size of 500. - `moon fmt`, `moon info`, and `moon check` completed without warnings. - `moon test` passed all 8422 tests, including the workspace-only generated conformance suites and legacy-package compatibility tests. - The five compatibility `.mbti` files are identical to the interfaces at commit `f3e356f1`. - `moon package --list` and zip integrity checks passed for all six published modules. No archive contains a conformance path. - Archive sizes are 16 KB (`unicode`), 137 KB (`ucd`), 15 KB (`normalization`), 10 KB (`punycode`), 27 KB (`bidi`), and 107 KB (`idna`). ## Rollback The split is delivered in coherent commits. Before publication, rollback is a normal Git revert to the original single-module layout; no external state or stored data migration is involved.