0c D512 Zhuyin Candidate Reranker, Tone-Optional

Project website: 0c.biggo.com

A neural candidate reranker for a Traditional Chinese Zhuyin (Bopomofo) input method. Given confirmed preceding text (up to 64 characters), Zhuyin readings, and supplied candidates, the model computes log P(candidate | context, reading). These scores can be combined with an input-method engine's scores to decide whether to override its default candidate. A candidate source is required; the model does not generate candidates itself.

Tones are optional. Each syllable may carry its tone mark, omit the tone entirely (unknown tone), or mix both within one reading, so the same model serves full-tone, toneless, and partially toned typing. Without a tone mark a syllable means first tone, as in earlier releases; append ? (or use toneless()) to mark a tone that was not typed.

  • Architecture: a dedicated causal Transformer decoder, 8 layers, hidden size 512, 8 attention heads, FFN size 768, pretrained from random initialization, further pretrained on additional text with tone-optional reading tasks, and then fine-tuned for candidate ranking
  • Format: ONNX (opset 18), CPU inference
  • Vocabulary: 27,857 characters and 416 Zhuyin bases (vocab.json)

Files

File Description
model.onnx Candidate scoring model; SHA-256 4a80caf18acbaddde10c908dcbf35919908d616e94618cdde71fc3409c969261
vocab.json Special tokens, character vocabulary, and Zhuyin bases; SHA-256 556dffaffbf4c05211a846dcee55ef735cea68c5754f9bffcc6f9588124172b3
manifest.json Operating points, input limits, and file hashes
q4/ 4-bit weight-quantized version of the same model (see 4-bit Version); model.onnx SHA-256 82d3ef3c1d23e887390674816bb3517fdb7725211dec7c7f0714029bc4c10c4e, model.onnx.data SHA-256 15498b930254ca1c91bd08aa9ba1400f85e6159830a9da664eff995482730ca0

Installation

Requires Python 3.9 or later. The examples use CPU inference.

pip install onnxruntime numpy huggingface_hub

The first run downloads the model from Hugging Face (approximately 130 MB; the 4-bit version in q4/ is not downloaded). Subsequent runs use the local cache.

Usage

Save the complete example below as rank.py and run it. Supply one string per Zhuyin syllable (for example, ㄍㄨㄥ and ㄩㄢˊ, or ㄩㄢ? when the tone was not typed). Each candidate must have exactly one character per syllable.

  • With supplied candidates only: use rank() to order homophonic candidates by model score and find the best match for the context.
  • With an input-method engine: use choose() to combine engine and model scores, overriding the engine default only when the required score margin is met.
import json

import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download

path = snapshot_download("Funmula/0c-sequence-d512", allow_patterns=["model.onnx", "vocab.json", "manifest.json"])
manifest = json.load(open(f"{path}/manifest.json", encoding="utf-8"))
vocab = json.load(open(f"{path}/vocab.json", encoding="utf-8"))
CHARS = {c: i + 8 for i, c in enumerate(vocab["characters"])}   # IDs 0–7 are special tokens; 4 = UNK
BASES = {b: i + 2 for i, b in enumerate(vocab["bases"])}
TONES = {"ˊ": 2, "ˇ": 3, "ˋ": 4, "˙": 5}                         # No tone mark = first tone
UNKNOWN = "?"                                                    # Trailing "?" = tone not typed (tone id 0)
NEUTRAL = set(manifest.get("neutral_tolerant_last", []))        # Final particles are often typed with first tone
session = ort.InferenceSession(f"{path}/model.onnx", providers=["CPUExecutionProvider"])


def syllable(s):
    if s.endswith(UNKNOWN):
        return BASES[s[:-1]], 0
    base = s[:-1] if s[-1] in TONES else s
    return BASES[base], TONES.get(s[-1], 1)


def toneless(reading):
    # Mark every syllable as tone-unknown, e.g. for users who type without tones.
    return [s.rstrip("ˊˇˋ˙" + UNKNOWN) + UNKNOWN for s in reading]


def log_prob(context, reading, candidate):
    # [BOS] context [SEP] reading×n [OUT] first n−1 candidate characters.
    # Return log P(candidate | context, reading).
    head = [1] + [CHARS.get(c, 4) for c in context[-64:]] + [2]
    sounds = [syllable(s) for s in reading]
    cand = [CHARS[c] for c in candidate]   # Unknown candidate characters raise KeyError for caller fallback
    n, out = len(cand), len(head) + len(sounds)
    ids = head + [0] * n + [3] + cand[:-1]
    bases = [0] * len(head) + [b for b, _ in sounds] + [0] * n
    tones = [0] * len(head) + [t for _, t in sounds] + [0] * n
    types = [0] * len(head) + [1] * n + [0] + [2] * (n - 1)
    L = len(ids)
    row = lambda x: np.array([x], dtype=np.int64)
    feeds = {"ids": row(ids), "bases": row(bases), "tones": row(tones), "types": row(types),
             "positions": row(list(range(L))), "mask": np.tril(np.ones((L, L), dtype=bool))[None, None],
             "proj_pos": np.arange(out, L, dtype=np.int64), "target_node": row(list(range(n))),
             "target_ids": row([x - 6 for x in cand])}
    return float(session.run(["scores"], feeds)[0][0])


def score(context, reading, candidate):
    s = log_prob(context, reading, candidate)
    last = reading[-1]
    if last.endswith(UNKNOWN):   # An unknown final tone already covers the neutral tone
        return s
    base = last[:-1] if last[-1] in TONES else last
    if base in NEUTRAL and last[-1] != "˙":   # Also score the final syllable with neutral tone and keep the higher score
        s = max(s, log_prob(context, reading[:-1] + [base + "˙"], candidate))
    return s


def choose(context, reading, candidates, engine_scores, w=manifest["weight"], tau=manifest["margin"]):
    # candidates[0] is the engine default; engine_scores are engine log scores.
    # Return the selected candidate index.
    try:
        neural = [score(context, reading, c) for c in candidates]
    except KeyError:
        return 0
    fused = [e + w * s for e, s in zip(engine_scores, neural)]
    best = max(range(len(fused)), key=fused.__getitem__)
    return best if best != 0 and fused[best] - fused[0] > tau else 0


def rank(context, reading, candidates):
    # Rank supplied homophonic candidates by model score alone, highest first.
    return sorted(candidates, key=lambda c: score(context, reading, c), reverse=True)


print(rank("下週就是", ["ㄑㄧˊ", "ㄓㄨㄥ", "ㄎㄠˇ"], ["其中考", "期中考"]))   # ['期中考', '其中考']
print(rank("", ["ㄐㄧㄣ", "ㄊㄧㄢ", "ㄊㄧㄢ", "ㄑㄧˋ", "ㄏㄣˇ", "ㄏㄠˇ"],
           ["金天天氣很好", "今天天氣很好", "今天天器很好"]))                   # ['今天天氣很好', ...]

# Tones are optional: mark untyped tones with "?" (or use toneless()); tones may be mixed per syllable.
print(rank("下週就是", toneless(["ㄑㄧ", "ㄓㄨㄥ", "ㄎㄠ"]), ["其中考", "期中考"]))   # ['期中考', '其中考']
print(rank("這家餐廳的", ["ㄘㄞ?", "ㄏㄣˇ", "ㄏㄠ?", "ㄔ"], ["才很好吃", "菜很好吃"]))   # ['菜很好吃', '才很好吃']

# With an input-method engine, override candidates[0] only when the fused score warrants it.
candidates = ["公元", "公園"]
print(candidates[choose("今天天氣很好我們去", ["ㄍㄨㄥ", "ㄩㄢˊ"], candidates, [-7.0, -7.5])])   # 公園

4-bit Version

q4/ contains the same model with 4-bit weights, for devices with less memory. It takes the same inputs and produces the same kind of scores, so the example above works after changing one line:

path = snapshot_download("Funmula/0c-sequence-d512", allow_patterns=["q4/*"]) + "/q4"   # approximately 86 MB

Keep model.onnx and model.onnx.data in the same directory; ONNX Runtime loads the weights from the data file. q4/ has its own vocab.json (identical to the root copy) and manifest.json (same operating points, plus a quantization field).

  • Quantization: round-to-nearest 4-bit weights in blocks of 16, symmetric, for all Transformer linear layers and the output projection (MatMulNBits, ONNX Runtime contrib operator, accuracy_level 1). Input embeddings stay float32 in the data file, so they can be memory-mapped.
  • Requires ONNX Runtime 1.20 or later (tested with 1.20.1 and 1.23.2; 1.17 and 1.18 cannot load it). Other ONNX runtimes without the com.microsoft operators cannot run it.
  • Size and memory: 86 MB on disk instead of 131 MB. In our Android measurements, resident private memory grew by about 45 MB after loading, compared with about 153 MB for the float32 model; speed was about the same.
  • Accuracy: on a 6,000-case development set per typing mode, using the float32 model as the reference, whole-target accuracy changed by −0.03 percentage points for full tones, 0.00 for partial tones, and −0.38 percentage points for toneless input (95% interval −0.73 to −0.05). Use the float32 model when memory allows.
  • Scores differ slightly from the float32 model, so a few close decisions change. Recalibrating w and τ is not required; the operating points in manifest.json are unchanged.

Operating Points and Limitations

  • Fused score = engine score + w × model score, with w = 2. Override only if another candidate has the highest score and exceeds the engine default by more than τ = 0.
  • Supported operating range: 2–32 syllables, at most 16 candidates, and the last 64 context characters.
  • If any candidate contains a character outside the vocabulary, choose() retains the engine default for the entire case.
  • Training candidates came from the 0c dictionary (dictionary_sha256 = 517e6c8bb5cbb145c0554edc3cc91c2b55fc753dd42e11194d83f6b0e119aea3). Calibrate w and τ again when using a different engine's score scale.
  • Shared candidate prefixes can also be scored together as a tree, using an ancestor attention mask and multiple target_node/target_ids rows. Scores match individual candidate scoring.
  • An unknown tone is encoded as tone id 0 (tone_policy in manifest.json). A final syllable with an unknown tone is not rescored as a neutral tone.

License and Release Scope

Model weights and the accompanying inference examples are released under the MIT license (see LICENSE). This release covers this D512 checkpoint only. Training code and training materials are not included.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support