CLI Reference Overview

The litsea CLI provides commands for word segmentation, model training, and text processing.

The CLI binary is built with the library’s remote_model feature enabled, so http(s):// model URIs work out of the box – unlike the litsea library itself, whose default features do not include remote loading.

Usage

litsea <COMMAND> [OPTIONS] [ARGS]

Commands

Command	Description
`extract`	Extract features from a corpus for training
`train`	Train a word segmentation model
`segment`	Segment text into words using a trained model
`evaluate`	Evaluate a model against a held-out gold corpus

Global Options

Option	Description
`-h`, `--help`	Show help information
`-V`, `--version`	Show version number

Typical Workflow

AdaBoost Workflow (Word Segmentation Only)

flowchart LR
    A["1. scripts/download_udtreebank.sh"] --> B["2. scripts/corpus_udtreebank.sh"]
    B --> C["3. litsea extract"]
    C --> D["4. litsea train"]
    D --> E["5. litsea segment"]

Download a UD Treebank: conllu_file=$(bash scripts/download_udtreebank.sh -l ja -o /tmp)
Convert to corpus format: bash scripts/corpus_udtreebank.sh "$conllu_file" corpus.txt
Extract features: litsea extract -l japanese corpus.txt features.txt
Train a model: litsea train -t 0.0001 -i 20000 features.txt model.model
Segment text: echo "text" | litsea segment -l japanese model.model

Two-Stage Workflow (Word Segmentation with POS Tagging)

flowchart LR
    A["1. scripts/download_udtreebank.sh"] --> B["2. scripts/corpus_udtreebank.sh -p"]
    B --> C["3. litsea extract --pos"]
    C --> D["4. litsea train --pos"]
    D --> E["5. litsea segment --pos"]

Download a UD Treebank: conllu_file=$(bash scripts/download_udtreebank.sh -l ja -o /tmp)
Convert to POS corpus format: bash scripts/corpus_udtreebank.sh -p "$conllu_file" pos_corpus.txt
Extract two-stage features: litsea extract --pos -l japanese pos_corpus.txt features_prefix
Train a two-stage model: litsea train --pos --num-epochs 50 features_prefix model.model
Segment with POS tags: echo "text" | litsea segment --pos -l japanese model.model

See Two-Stage Tagging for the architecture, and Training Models or train for the full flag reference.

Keyboard shortcuts