unicode

Branch: main

Switch branches or tags

Branch: main
Go to file t

413 files indexed

History
Code

docs/plans/2026-07-15-module-split.md

Unicode Module Split

Status: Validated

Outcome

Split the published moonbit-community/unicode module into independently consumable modules so users only download the Unicode functionality they use:

  • moonbit-community/ucd
  • moonbit-community/normalization
  • moonbit-community/punycode
  • moonbit-community/bidi
  • moonbit-community/idna

Keep moonbit-community/unicode as a compatibility umbrella whose existing package paths forward to the new modules.

Boundaries

  • Each feature module owns its public types and implementation data.
  • moonbit-community/ucd/data exposes the low-level table lookups required by normalization and IDNA; user-facing case and category APIs remain at moonbit-community/ucd.
  • BidiClass is owned by the public moonbit-community/bidi package rather than an internal package.
  • moonbit-community/idna depends on UCD, normalization, Punycode, and Bidi.
  • No persistence, wire-format, security-policy, or third-party dependency changes are included.

Tests and generation

Unit and fuzz tests stay with their owning feature modules. Large generated Unicode, IDNA, and Bidi conformance fixtures move to a workspace-only moonbit-community/unicode-conformance module. They remain part of local and CI test runs but are excluded from published .mooncakes archives.

The existing moonbit-community/unicode-tools module remains workspace-only and is updated to emit data, interfaces, and conformance fixtures at the new paths.

Compatibility

The umbrella module keeps the existing public package paths:

  • moonbit-community/unicode
  • moonbit-community/unicode/normalization
  • moonbit-community/unicode/punycode
  • moonbit-community/unicode/bidi
  • moonbit-community/unicode/idna

These packages are thin facades over the new modules. New consumers should depend directly on the feature module they need.

Validation

  1. Run moon fmt and moon info for the workspace.
  2. Run moon check and moon test for every feature module and for the whole workspace.
  3. Regenerate all Unicode data and conformance fixtures, then repeat checks.
  4. Run moon package --list in every published module.
  5. Record archive sizes and verify conformance fixtures are absent.

Validation evidence

  • moon run --target native tools/gen all completed with cached Unicode 16.0.0 source data and the default Bidi part size of 500.
  • moon fmt, moon info, and moon check completed without warnings.
  • moon test passed all 8422 tests, including the workspace-only generated conformance suites and legacy-package compatibility tests.
  • The five compatibility .mbti files are identical to the interfaces at commit f3e356f1.
  • moon package --list and zip integrity checks passed for all six published modules. No archive contains a conformance path.
  • Archive sizes are 16 KB (unicode), 137 KB (ucd), 15 KB (normalization), 10 KB (punycode), 27 KB (bidi), and 107 KB (idna).

Rollback

The split is delivered in coherent commits. Before publication, rollback is a normal Git revert to the original single-module layout; no external state or stored data migration is involved.

1# Unicode Module Split23Status: Validated45## Outcome67Split the published `moonbit-community/unicode` module into independently consumable8modules so users only download the Unicode functionality they use:910- `moonbit-community/ucd`11- `moonbit-community/normalization`12- `moonbit-community/punycode`13- `moonbit-community/bidi`14- `moonbit-community/idna`1516Keep `moonbit-community/unicode` as a compatibility umbrella whose existing package17paths forward to the new modules.1819## Boundaries2021- Each feature module owns its public types and implementation data.22- `moonbit-community/ucd/data` exposes the low-level table lookups required by23  normalization and IDNA; user-facing case and category APIs remain at24  `moonbit-community/ucd`.25- `BidiClass` is owned by the public `moonbit-community/bidi` package rather than an26  internal package.27- `moonbit-community/idna` depends on UCD, normalization, Punycode, and Bidi.28- No persistence, wire-format, security-policy, or third-party dependency29  changes are included.3031## Tests and generation3233Unit and fuzz tests stay with their owning feature modules. Large generated34Unicode, IDNA, and Bidi conformance fixtures move to a workspace-only35`moonbit-community/unicode-conformance` module. They remain part of local and CI test36runs but are excluded from published `.mooncakes` archives.3738The existing `moonbit-community/unicode-tools` module remains workspace-only and is39updated to emit data, interfaces, and conformance fixtures at the new paths.4041## Compatibility4243The umbrella module keeps the existing public package paths:4445- `moonbit-community/unicode`46- `moonbit-community/unicode/normalization`47- `moonbit-community/unicode/punycode`48- `moonbit-community/unicode/bidi`49- `moonbit-community/unicode/idna`5051These packages are thin facades over the new modules. New consumers should52depend directly on the feature module they need.5354## Validation55561. Run `moon fmt` and `moon info` for the workspace.572. Run `moon check` and `moon test` for every feature module and for the whole58   workspace.593. Regenerate all Unicode data and conformance fixtures, then repeat checks.604. Run `moon package --list` in every published module.615. Record archive sizes and verify conformance fixtures are absent.6263## Validation evidence6465- `moon run --target native tools/gen all` completed with cached Unicode 16.0.066  source data and the default Bidi part size of 500.67- `moon fmt`, `moon info`, and `moon check` completed without warnings.68- `moon test` passed all 8422 tests, including the workspace-only generated69  conformance suites and legacy-package compatibility tests.70- The five compatibility `.mbti` files are identical to the interfaces at71  commit `f3e356f1`.72- `moon package --list` and zip integrity checks passed for all six published73  modules. No archive contains a conformance path.74- Archive sizes are 16 KB (`unicode`), 137 KB (`ucd`), 15 KB75  (`normalization`), 10 KB (`punycode`), 27 KB (`bidi`), and 107 KB (`idna`).7677## Rollback7879The split is delivered in coherent commits. Before publication, rollback is a80normal Git revert to the original single-module layout; no external state or81stored data migration is involved.82