unicode
Cross-language benchmark
Compares this repository's modules with the libraries other languages use for the same jobs. Run from the repository root:
moon run --target native bench/xlang
moon run --target native bench/xlang -- --rounds 1 --group bidi
moon run --target native bench/xlang -- --impl moonbit-native,rust --detail
moon run --target native bench/xlang -- --base HEAD^
The runner (the MoonBit package in this directory) prepares the inputs, builds
every harness, runs them one at a time for three rounds, and prints a Markdown
report. Nothing is written to the source tree: inputs, build products,
results.tsv, report.md and results.json (for the dashboard) go to
_build/xlang/. A full run takes a few minutes.
| Option | Meaning |
|---|---|
--rounds N |
Rounds to run; each case keeps its fastest round (default 3) |
--impl A,B |
Only these harnesses (default: every one that can be set up) |
--group G |
Only norm, seg, bidi, idna, puny or case |
--detail |
Also report every case, not only the mean per operation |
--base REV |
Compare the MoonBit harnesses with those of commit REV instead; see below |
Harnesses
Each subdirectory is a standalone program in its own language. Nothing is called across languages: every program reads the same input files, times the same cases with the same protocol, and prints rows in the same format.
| Harness | Directory | Measures | Needs |
|---|---|---|---|
moonbit-native, moonbit-wasm-gc, moonbit-wasm, moonbit-js |
moonbit/ |
This checkout, per backend | moon; node for js |
icu4c |
icu4c/ |
ICU4C, on UTF-16 | A C++17 compiler and ICU4C |
rust |
rust/ |
unicode-normalization, unicode-segmentation, unicode-bidi, idna, unicode-general-category, std, and ICU4X on UTF-8 and UTF-16 |
cargo |
go |
go/ |
golang.org/x/text, golang.org/x/net/idna, rivo/uniseg, unicode |
go 1.26 or later |
node |
node/ |
String.prototype.normalize and Intl.Segmenter (ICU4C behind V8), url.domainToASCII (ada), the bundled punycode |
node |
python |
python/ |
unicodedata for normalization, str.casefold |
python3 |
A harness whose toolchain, runtime or library is missing is reported as
skipped and the comparison continues without it; the run fails only when no
harness produces a result. ICU4C is looked up in ICU_ROOT, then in
Homebrew's keg-only icu4c, then on the compiler's default paths; CXX
selects the compiler. Rust and Go dependencies are pinned by Cargo.lock and
go.sum.
Protocol
Every harness doubles the iteration count of a case until one sample takes at least 5 ms, then reports the fastest of 5 samples, counting that one, as
library<TAB>group/operation/input<TAB>ns per call<TAB>units<TAB>check
library is empty when the harness measures a single library, units is the
number of code points processed per call (1 for per-call cases), and check
is a value every implementation must agree on, such as the number of code
points of the result, the grapheme cluster count or the sum of the Bidi
levels (- when there is nothing comparable). File IO, decoding and input
preparation are outside the timed calls. The report lists the cases on which
implementations disagree.
A harness must keep every result where it could be read later, not only fold
its length into the sink: V8 drops a string that only has its length read,
since it can compute the length without building the string, and LLVM can drop
an allocation it sees unused. The MoonBit and Node harnesses store each result
in a global; the Rust harness passes it through black_box.
The inputs are defined once, in inputs.mbt; harnesses list the input
directories rather than naming files. Normalization uses the 14 ICU normperf
corpora committed for bench/normperf, as they are
and converted to NFD. The case group has short phrases repeated, three of
those corpora (Latin and Russian names, English prose) and the Latin names
in uppercase, where nearly every letter has a mapping to apply.
To add a harness, write a program that takes <inputs> [group] and prints
rows in this format, then add it to all_peers in peers.mbt.
Reading the report
Cells are nanoseconds per code point (per call for idna and puny), as the
geometric mean over an operation's inputs. The number in parentheses is the
time relative to moonbit-native on the same cases.
- MoonBit, ICU4C and Node work on UTF-16; the Rust and Go libraries work on UTF-8, which is why ICU4X is measured on both.
- For input that is already normalized, MoonBit, ICU4X and Go can return the
input unchanged, while
unicode-normalizationalways builds a new string. Thenfdandnfd>nfcoperations force real work. Python answers normalization of ASCII-only strings in constant time, which pulls its geometric means down. - The IDNA options are only approximately aligned: UTS #46 non-transitional processing without STD3 rules, with each library's own defaults otherwise.
unisegimplements an older Unicode version, so its grapheme cluster count differs on Devanagari text.case/full_uppercompares MoonBit'suppercase, which builds one string for the whole text, with Rust'schar::to_uppercaseiterator, which is counted without building anything.case/foldis full default case folding (statuses C and F, no Turkic mappings): ICU4C'su_strFoldCaseon UTF-16, ICU4X'sfold_string, Go'scases.Fold()and Python'sstr.casefold. ICU4X and Go return the input when folding changes nothing, as on the CJK input; the others always build a new string, and Python has a dedicated loop for ASCII-only strings. Node and the Rust standard library have no full case folding, and only ICU4C hasnfkc_casefold.- ICU4C's per-character functions are calls into a shared library, and every Node case crosses from JavaScript into the runtime.
- Other processes disturb the measurement. The report's noise table shows how much the rounds disagreed; treat differences of that size as equal.
Comparing two commits
--base REV checks REV out in _build/xlang/base, copies this checkout's
MoonBit harness and bench/moon.mod into it, and runs the MoonBit harnesses
of both commits in turns: in odd rounds the base goes first, in even rounds the head. Both sides
run the same harness, so a change to moonbit/main.mbt does not show up as a
change to the library. Only the MoonBit harnesses take part. The report,
compare.md, gives the change in time per operation and backend, and
compare.json holds the costs of every case on both sides.
Dashboard
site/ is a static page that shows both kinds of run. The benchmark
workflow (.github/workflows/bench.yml) runs --base HEAD^ on every push to
main, appends the result to history.json on the bench-data branch, keeps
the full comparison in runs/<commit>.json there, and publishes site/ with
the history to GitHub Pages. GitHub's runners are shared machines, so only
these same-runner comparisons are recorded from CI, never absolute times. A
comparison runs two rounds.
Pushes that only change documentation (*.md, docs/), tests (*_test.mbt,
*_wbtest.mbt, tests/), tools/ or other workflows start no run, since they
cannot change the library's speed, and leave no entry in the history. Pushes
that only change site/ republish the page without comparing. Running the
workflow by hand republishes the page, and compares too when its compare
input is set.
Ticking a stretch of consecutive merges in the history adds a row that
combines them: the change from the base of the oldest to the head of the
newest. Runs land on runners of different speeds, so it chains each case's
ratio of head to base from runs/<commit>.json rather than comparing times
across runs, and takes the geometric mean over the cases that every merge in
the stretch measured. The page keeps the stretch in its address
(#range=<base>..<head>). For a figure measured in one go, run
--base <older commit>.
The cross-language part of the page reads site/snapshot.json, which is
updated by hand from a full run on a quiet machine:
moon run --target native bench/xlang
cp _build/xlang/results.json bench/xlang/site/snapshot.json
Commit it; the push to main republishes the page. To look at the page
locally, serve site/ over HTTP (it fetches its data), for example with
python3 -m http.server -d bench/xlang/site. Without a history.json next to
it, the history section says there are no runs yet.
1# Cross-language benchmark23Compares this repository's modules with the libraries other languages use for4the same jobs. Run from the repository root:56```sh7moon run --target native bench/xlang8moon run --target native bench/xlang -- --rounds 1 --group bidi9moon run --target native bench/xlang -- --impl moonbit-native,rust --detail10moon run --target native bench/xlang -- --base HEAD^11```1213The runner (the MoonBit package in this directory) prepares the inputs, builds14every harness, runs them one at a time for three rounds, and prints a Markdown15report. Nothing is written to the source tree: inputs, build products,16`results.tsv`, `report.md` and `results.json` (for the dashboard) go to17`_build/xlang/`. A full run takes a few minutes.1819| Option | Meaning |20|---|---|21| `--rounds N` | Rounds to run; each case keeps its fastest round (default 3) |22| `--impl A,B` | Only these harnesses (default: every one that can be set up) |23| `--group G` | Only `norm`, `seg`, `bidi`, `idna`, `puny` or `case` |24| `--detail` | Also report every case, not only the mean per operation |25| `--base REV` | Compare the MoonBit harnesses with those of commit `REV` instead; see below |2627## Harnesses2829Each subdirectory is a standalone program in its own language. Nothing is30called across languages: every program reads the same input files, times the31same cases with the same protocol, and prints rows in the same format.3233| Harness | Directory | Measures | Needs |34|---|---|---|---|35| `moonbit-native`, `moonbit-wasm-gc`, `moonbit-wasm`, `moonbit-js` | `moonbit/` | This checkout, per backend | `moon`; `node` for `js` |36| `icu4c` | `icu4c/` | ICU4C, on UTF-16 | A C++17 compiler and ICU4C |37| `rust` | `rust/` | `unicode-normalization`, `unicode-segmentation`, `unicode-bidi`, `idna`, `unicode-general-category`, `std`, and ICU4X on UTF-8 and UTF-16 | `cargo` |38| `go` | `go/` | `golang.org/x/text`, `golang.org/x/net/idna`, `rivo/uniseg`, `unicode` | `go` 1.26 or later |39| `node` | `node/` | `String.prototype.normalize` and `Intl.Segmenter` (ICU4C behind V8), `url.domainToASCII` (ada), the bundled `punycode` | `node` |40| `python` | `python/` | `unicodedata` for normalization, `str.casefold` | `python3` |4142A harness whose toolchain, runtime or library is missing is reported as43skipped and the comparison continues without it; the run fails only when no44harness produces a result. ICU4C is looked up in `ICU_ROOT`, then in45Homebrew's keg-only `icu4c`, then on the compiler's default paths; `CXX`46selects the compiler. Rust and Go dependencies are pinned by `Cargo.lock` and47`go.sum`.4849## Protocol5051Every harness doubles the iteration count of a case until one sample takes at52least 5 ms, then reports the fastest of 5 samples, counting that one, as5354```55library<TAB>group/operation/input<TAB>ns per call<TAB>units<TAB>check56```5758`library` is empty when the harness measures a single library, `units` is the59number of code points processed per call (1 for per-call cases), and `check`60is a value every implementation must agree on, such as the number of code61points of the result, the grapheme cluster count or the sum of the Bidi62levels (`-` when there is nothing comparable). File IO, decoding and input63preparation are outside the timed calls. The report lists the cases on which64implementations disagree.6566A harness must keep every result where it could be read later, not only fold67its length into the sink: V8 drops a string that only has its length read,68since it can compute the length without building the string, and LLVM can drop69an allocation it sees unused. The MoonBit and Node harnesses store each result70in a global; the Rust harness passes it through `black_box`.7172The inputs are defined once, in `inputs.mbt`; harnesses list the input73directories rather than naming files. Normalization uses the 14 ICU normperf74corpora committed for [`bench/normperf`](../normperf/README.md), as they are75and converted to NFD. The `case` group has short phrases repeated, three of76those corpora (Latin and Russian names, English prose) and the Latin names77in uppercase, where nearly every letter has a mapping to apply.7879To add a harness, write a program that takes `<inputs> [group]` and prints80rows in this format, then add it to `all_peers` in `peers.mbt`.8182## Reading the report8384Cells are nanoseconds per code point (per call for `idna` and `puny`), as the85geometric mean over an operation's inputs. The number in parentheses is the86time relative to `moonbit-native` on the same cases.8788- MoonBit, ICU4C and Node work on UTF-16; the Rust and Go libraries work on89 UTF-8, which is why ICU4X is measured on both.90- For input that is already normalized, MoonBit, ICU4X and Go can return the91 input unchanged, while `unicode-normalization` always builds a new string.92 The `nfd` and `nfd>nfc` operations force real work. Python answers93 normalization of ASCII-only strings in constant time, which pulls its94 geometric means down.95- The IDNA options are only approximately aligned: UTS #46 non-transitional96 processing without STD3 rules, with each library's own defaults otherwise.97- `uniseg` implements an older Unicode version, so its grapheme cluster count98 differs on Devanagari text.99- `case/full_upper` compares MoonBit's `uppercase`, which builds one string100 for the whole text, with Rust's `char::to_uppercase` iterator, which is101 counted without building anything.102- `case/fold` is full default case folding (statuses C and F, no Turkic103 mappings): ICU4C's `u_strFoldCase` on UTF-16, ICU4X's `fold_string`, Go's104 `cases.Fold()` and Python's `str.casefold`. ICU4X and Go return the input105 when folding changes nothing, as on the CJK input; the others always build106 a new string, and Python has a dedicated loop for ASCII-only strings. Node107 and the Rust standard library have no full case folding, and only ICU4C108 has `nfkc_casefold`.109- ICU4C's per-character functions are calls into a shared library, and every110 Node case crosses from JavaScript into the runtime.111- Other processes disturb the measurement. The report's noise table shows how112 much the rounds disagreed; treat differences of that size as equal.113114## Comparing two commits115116`--base REV` checks `REV` out in `_build/xlang/base`, copies this checkout's117MoonBit harness and `bench/moon.mod` into it, and runs the MoonBit harnesses118of both commits in turns: in odd rounds the base goes first, in even rounds the head. Both sides119run the same harness, so a change to `moonbit/main.mbt` does not show up as a120change to the library. Only the MoonBit harnesses take part. The report,121`compare.md`, gives the change in time per operation and backend, and122`compare.json` holds the costs of every case on both sides.123124## Dashboard125126`site/` is a static page that shows both kinds of run. The `benchmark`127workflow (`.github/workflows/bench.yml`) runs `--base HEAD^` on every push to128main, appends the result to `history.json` on the `bench-data` branch, keeps129the full comparison in `runs/<commit>.json` there, and publishes `site/` with130the history to GitHub Pages. GitHub's runners are shared machines, so only131these same-runner comparisons are recorded from CI, never absolute times. A132comparison runs two rounds.133134Pushes that only change documentation (`*.md`, `docs/`), tests (`*_test.mbt`,135`*_wbtest.mbt`, `tests/`), `tools/` or other workflows start no run, since they136cannot change the library's speed, and leave no entry in the history. Pushes137that only change `site/` republish the page without comparing. Running the138workflow by hand republishes the page, and compares too when its `compare`139input is set.140141Ticking a stretch of consecutive merges in the history adds a row that142combines them: the change from the base of the oldest to the head of the143newest. Runs land on runners of different speeds, so it chains each case's144ratio of head to base from `runs/<commit>.json` rather than comparing times145across runs, and takes the geometric mean over the cases that every merge in146the stretch measured. The page keeps the stretch in its address147(`#range=<base>..<head>`). For a figure measured in one go, run148`--base <older commit>`.149150The cross-language part of the page reads `site/snapshot.json`, which is151updated by hand from a full run on a quiet machine:152153```sh154moon run --target native bench/xlang155cp _build/xlang/results.json bench/xlang/site/snapshot.json156```157158Commit it; the push to main republishes the page. To look at the page159locally, serve `site/` over HTTP (it fetches its data), for example with160`python3 -m http.server -d bench/xlang/site`. Without a `history.json` next to161it, the history section says there are no runs yet.162# Cross-language benchmark Compares this repository's modules with the libraries other languages use for the same jobs. Run from the repository root: ```sh moon run --target native bench/xlang moon run --target native bench/xlang -- --rounds 1 --group bidi moon run --target native bench/xlang -- --impl moonbit-native,rust --detail moon run --target native bench/xlang -- --base HEAD^ ``` The runner (the MoonBit package in this directory) prepares the inputs, builds every harness, runs them one at a time for three rounds, and prints a Markdown report. Nothing is written to the source tree: inputs, build products, `results.tsv`, `report.md` and `results.json` (for the dashboard) go to `_build/xlang/`. A full run takes a few minutes. | Option | Meaning | |---|---| | `--rounds N` | Rounds to run; each case keeps its fastest round (default 3) | | `--impl A,B` | Only these harnesses (default: every one that can be set up) | | `--group G` | Only `norm`, `seg`, `bidi`, `idna`, `puny` or `case` | | `--detail` | Also report every case, not only the mean per operation | | `--base REV` | Compare the MoonBit harnesses with those of commit `REV` instead; see below | ## Harnesses Each subdirectory is a standalone program in its own language. Nothing is called across languages: every program reads the same input files, times the same cases with the same protocol, and prints rows in the same format. | Harness | Directory | Measures | Needs | |---|---|---|---| | `moonbit-native`, `moonbit-wasm-gc`, `moonbit-wasm`, `moonbit-js` | `moonbit/` | This checkout, per backend | `moon`; `node` for `js` | | `icu4c` | `icu4c/` | ICU4C, on UTF-16 | A C++17 compiler and ICU4C | | `rust` | `rust/` | `unicode-normalization`, `unicode-segmentation`, `unicode-bidi`, `idna`, `unicode-general-category`, `std`, and ICU4X on UTF-8 and UTF-16 | `cargo` | | `go` | `go/` | `golang.org/x/text`, `golang.org/x/net/idna`, `rivo/uniseg`, `unicode` | `go` 1.26 or later | | `node` | `node/` | `String.prototype.normalize` and `Intl.Segmenter` (ICU4C behind V8), `url.domainToASCII` (ada), the bundled `punycode` | `node` | | `python` | `python/` | `unicodedata` for normalization, `str.casefold` | `python3` | A harness whose toolchain, runtime or library is missing is reported as skipped and the comparison continues without it; the run fails only when no harness produces a result. ICU4C is looked up in `ICU_ROOT`, then in Homebrew's keg-only `icu4c`, then on the compiler's default paths; `CXX` selects the compiler. Rust and Go dependencies are pinned by `Cargo.lock` and `go.sum`. ## Protocol Every harness doubles the iteration count of a case until one sample takes at least 5 ms, then reports the fastest of 5 samples, counting that one, as ``` library<TAB>group/operation/input<TAB>ns per call<TAB>units<TAB>check ``` `library` is empty when the harness measures a single library, `units` is the number of code points processed per call (1 for per-call cases), and `check` is a value every implementation must agree on, such as the number of code points of the result, the grapheme cluster count or the sum of the Bidi levels (`-` when there is nothing comparable). File IO, decoding and input preparation are outside the timed calls. The report lists the cases on which implementations disagree. A harness must keep every result where it could be read later, not only fold its length into the sink: V8 drops a string that only has its length read, since it can compute the length without building the string, and LLVM can drop an allocation it sees unused. The MoonBit and Node harnesses store each result in a global; the Rust harness passes it through `black_box`. The inputs are defined once, in `inputs.mbt`; harnesses list the input directories rather than naming files. Normalization uses the 14 ICU normperf corpora committed for [`bench/normperf`](../normperf/README.md), as they are and converted to NFD. The `case` group has short phrases repeated, three of those corpora (Latin and Russian names, English prose) and the Latin names in uppercase, where nearly every letter has a mapping to apply. To add a harness, write a program that takes `<inputs> [group]` and prints rows in this format, then add it to `all_peers` in `peers.mbt`. ## Reading the report Cells are nanoseconds per code point (per call for `idna` and `puny`), as the geometric mean over an operation's inputs. The number in parentheses is the time relative to `moonbit-native` on the same cases. - MoonBit, ICU4C and Node work on UTF-16; the Rust and Go libraries work on UTF-8, which is why ICU4X is measured on both. - For input that is already normalized, MoonBit, ICU4X and Go can return the input unchanged, while `unicode-normalization` always builds a new string. The `nfd` and `nfd>nfc` operations force real work. Python answers normalization of ASCII-only strings in constant time, which pulls its geometric means down. - The IDNA options are only approximately aligned: UTS #46 non-transitional processing without STD3 rules, with each library's own defaults otherwise. - `uniseg` implements an older Unicode version, so its grapheme cluster count differs on Devanagari text. - `case/full_upper` compares MoonBit's `uppercase`, which builds one string for the whole text, with Rust's `char::to_uppercase` iterator, which is counted without building anything. - `case/fold` is full default case folding (statuses C and F, no Turkic mappings): ICU4C's `u_strFoldCase` on UTF-16, ICU4X's `fold_string`, Go's `cases.Fold()` and Python's `str.casefold`. ICU4X and Go return the input when folding changes nothing, as on the CJK input; the others always build a new string, and Python has a dedicated loop for ASCII-only strings. Node and the Rust standard library have no full case folding, and only ICU4C has `nfkc_casefold`. - ICU4C's per-character functions are calls into a shared library, and every Node case crosses from JavaScript into the runtime. - Other processes disturb the measurement. The report's noise table shows how much the rounds disagreed; treat differences of that size as equal. ## Comparing two commits `--base REV` checks `REV` out in `_build/xlang/base`, copies this checkout's MoonBit harness and `bench/moon.mod` into it, and runs the MoonBit harnesses of both commits in turns: in odd rounds the base goes first, in even rounds the head. Both sides run the same harness, so a change to `moonbit/main.mbt` does not show up as a change to the library. Only the MoonBit harnesses take part. The report, `compare.md`, gives the change in time per operation and backend, and `compare.json` holds the costs of every case on both sides. ## Dashboard `site/` is a static page that shows both kinds of run. The `benchmark` workflow (`.github/workflows/bench.yml`) runs `--base HEAD^` on every push to main, appends the result to `history.json` on the `bench-data` branch, keeps the full comparison in `runs/<commit>.json` there, and publishes `site/` with the history to GitHub Pages. GitHub's runners are shared machines, so only these same-runner comparisons are recorded from CI, never absolute times. A comparison runs two rounds. Pushes that only change documentation (`*.md`, `docs/`), tests (`*_test.mbt`, `*_wbtest.mbt`, `tests/`), `tools/` or other workflows start no run, since they cannot change the library's speed, and leave no entry in the history. Pushes that only change `site/` republish the page without comparing. Running the workflow by hand republishes the page, and compares too when its `compare` input is set. Ticking a stretch of consecutive merges in the history adds a row that combines them: the change from the base of the oldest to the head of the newest. Runs land on runners of different speeds, so it chains each case's ratio of head to base from `runs/<commit>.json` rather than comparing times across runs, and takes the geometric mean over the cases that every merge in the stretch measured. The page keeps the stretch in its address (`#range=<base>..<head>`). For a figure measured in one go, run `--base <older commit>`. The cross-language part of the page reads `site/snapshot.json`, which is updated by hand from a full run on a quiet machine: ```sh moon run --target native bench/xlang cp _build/xlang/results.json bench/xlang/site/snapshot.json ``` Commit it; the push to main republishes the page. To look at the page locally, serve `site/` over HTTP (it fetches its data), for example with `python3 -m http.server -d bench/xlang/site`. Without a `history.json` next to it, the history section says there are no runs yet.