Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Node.js

litsea-nodejs exposes Litsea to Node.js 20+ through napi-rs. It is published to npm as litsea, with prebuilt native binaries for Linux, macOS, and Windows on x64 and arm64.

For browsers, use litsea-wasm instead.

Installation

npm install litsea

Getting a model

The package contains no models. Download one from the models/ directory and pass its path — see Pre-trained Models. The model file identifies its own kind, so hasPos reports what the loaded model can do and no flag is needed.

Segmentation

import { Segmenter } from 'litsea'

const seg = Segmenter.open('japanese', 'models/japanese.model')

seg.segment('これはテストです。')
// [ 'これ', 'は', 'テスト', 'です', '。' ]

The language name and its ISO 639-1 code are interchangeable ('ja', 'japanese').

For space-delimited languages the whitespace is returned as its own token, so the tokens always reconstruct the input:

Segmenter.open('ko', 'models/korean.model').segment('안녕하세요 반갑습니다')
// [ '안녕하세요', ' ', '반갑습니다' ]

POS tagging

const seg = Segmenter.open('japanese', 'models/japanese_pos.model')

seg.segmentWithPos('これはテストです。')
// [ { surface: 'これ', start: 0, end: 6, pos: 'PRON' },
//   { surface: 'は', start: 6, end: 9, pos: 'ADP' },
//   { surface: 'テスト', start: 9, end: 18, pos: 'NOUN' },
//   { surface: 'です', start: 18, end: 24, pos: 'AUX' },
//   { surface: '。', start: 24, end: 27, pos: 'PUNCT' } ]

start and end are byte offsets. JavaScript string indices are UTF-16 code units, so slice with a Buffer:

Buffer.from(text).subarray(token.start, token.end).toString()   // === token.surface

pos is undefined on tokens from segmentTokens, which does no tagging.

User dictionary

Every constructor takes two optional trailing arguments (#263): userWords, an array of words to emit as single tokens wherever they occur, and userWordsGate, a flip ratio in 0–1 that applies a word only where the model’s margin against it is small enough.

const seg = Segmenter.open('japanese', 'models/japanese.model', ['雪だるま', '新幹線'])

seg.segment('雪だるまを作る')
// [ '雪だるま', 'を', '作る' ]   (without the dictionary: [ '雪', 'だる', 'ま', 'を', '作る' ])

A registered word gets a boundary at each edge and none inside, overlapping occurrences are resolved leftmost-longest, and words of one character are ignored. With a POS model the word is tagged like any other word (雪だるま/NOUN with japanese_pos.model). Matching is context-free, so a short fragment registered as a word breaks every other word it occurs in; a userWordsGate of 0 applies a word only where the model already agrees with it. A gate outside 0–1 throws with the invalid_argument code — synchronously, also from fromUri. See Segmenter for the full semantics.

API

CallReturns
Segmenter.open(language, path, userWords?, userWordsGate?)A segmenter (synchronous)
Segmenter.fromBytes(language, buffer, userWords?, userWordsGate?)A segmenter (synchronous)
Segmenter.fromUri(language, uri, userWords?, userWordsGate?)Promise<Segmenter> — downloads off the event loop
segment(text)string[]
segmentBatch(texts)string[][]
segmentTokens(text)Token[] with byte offsets
segmentWithPos(text)Token[] with tags and offsets
segmentWithPosBatch(texts)Token[][]
new Extractor(language).extract(...)Promise<void>
new Extractor(language).extractTwoStage(...)Promise<void>
new Trainer(threshold, iterations, features).train(model, cancel?)Promise<BinaryMetrics>
new PerceptronTrainer(epochs, features).train(model, cancel?)Promise<MulticlassMetrics>
new TwoStageTrainer(epochs, prefix, dominance?).train(model, cancel?)Promise<TwoStageMetrics>

Type definitions are generated by napi-rs and shipped as index.d.ts.

Asynchronous by design

Downloading a model, extracting features, and training all return promises and run on libuv’s threadpool, so the event loop keeps turning. That is what makes cancellation useful:

import { CancelToken, Trainer } from 'litsea'

const cancel = new CancelToken()
setTimeout(() => cancel.cancel(), 60_000)

const metrics = await new Trainer(0.01, 100_000, 'features.txt').train('japanese.model', cancel)

Cancelling is not an error: training stops at its next check point, still writes the partially trained model, and resolves with its metrics. The binding never installs a signal handler.

Segmentation itself is synchronous: it is fast enough that a promise would cost more than the work.

A TwoStageTrainer can only be used once — training collapses stage 1 into an AdaBoost model, which consumes it. available reports the state, and a second train() rejects.

Errors

Every error carries a code, matching the error kinds the other bindings expose. Rejected promises carry the same codes as thrown errors.

err.codeRaised when
invalid_argumentUnknown language name, unknown feature set, reused trainer, userWordsGate outside 0–1
modelDownload failed, or the file is a legacy joint POS model
ioA file could not be read or written
parseThe model or training data is malformed
unsupportedThe scheme or operation is unavailable in this build
pos_unavailablePOS tagging requested from a segmentation-only model

Because napi::Status is a closed enum, the code reaches JavaScript two ways: synchronous calls use napi’s string-status error, and asynchronous ones rebuild the JavaScript Error object in Task::reject so the property survives the rejection.

Development

make test-litsea-nodejs    # cargo test + napi build + node --test
make lint-litsea-nodejs    # clippy
make build-litsea-nodejs   # release build

index.js and index.d.ts are generated by napi build and committed; CI rebuilds them and fails if the committed copies are stale. Their content comes from the templates of @napi-rs/cli, so package.json pins the CLI to an exact version: with a floating range, an upstream CLI release alone would make the committed copies stale, as 3.9.1 and 3.10.0 did in September 2026. Bumping the CLI therefore means regenerating both files in the same PR. The parity tests build the litsea CLI and compare the binding’s output against it.