RuleChef Live

Model calls turn into rules. A stream of 3,000 requests: sentences from European Court of Human Rights judgments (the public TAB anonymisation dataset), each asking for its dates and times, codes and reference numbers, and quantities. The current rule set tries each request first. If its gate says the rules cover the request, the rules answer and no model is called. Otherwise the teacher model, vela3-fast, answers, and its answer becomes a labelled example. After the first 50 examples and then every 100 new ones, RuleChef's search synthesis (no LLM) learns a new rule set from all of them on CPU.

The stream was run once offline with the code in this Space (sim.py); this page replays that run, so the rule sets and synthesis times are snapshots. The rules themselves run live in your browser, here and in the box at the bottom.

Speed 3,000

Share of requests answered by rules last 200 requests

Model calls cumulative

Latency per request mean of the last 200, ms

Rule set versions, scored on the holdout when deployed

Holdout: 400 sentences from other documents. Matching is RuleChef's TAB scorer (same text, same type). "Cascade" means rules where the gate lets them answer, teacher elsewhere. The last column compares with TAB's human annotation; the rules learn from the teacher, mistakes included.

Rules of the current version

Latest requests

Try your own text

The gate and the rules of the version at the slider position run in your browser. The teacher does not run on this static page: when the gate sends a request on, the page says so and shows what the rules found. For a holdout sentence it also shows the teacher's recorded answer.

How the numbers are computed

All numbers come from this demo's own run over public data; nothing is copied from a paper. The page aggregates the run's per-request log (data/replay.json) as you move the slider.

Stream and holdout. TAB documents are split by document: 3,000 sentences for the stream, 400 from other documents for the holdout. One request = one sentence of 40 to 400 characters.
Teacher. vela3-fast at revision 078c42f9, one span question per request with three label names asked at request time: "date or time", "code or reference number", "quantity". Its answers were computed once offline on CPU (16 processes with 2 threads each); the per-request time of that run is the teacher latency shown.
Rules. RuleChef search synthesis (synth="search", min support 2) on all labelled examples so far, with the teacher's answers as labels. A learned rule answers only if it matched at least 3 times on the labelled data with precision at least 0.8 against the teacher. Rule latency is the measured time of the gate plus RuleChef's Python executor on one CPU core.
Gate. Rules answer a request only when every trigger token (anything with a digit, plus words the teacher often labels) is covered by an answering rule or sits in a learned skip context. A rule answer with no spans ("nothing to extract") means the gate saw no uncovered trigger.
Spot checks and drift guard. Every 10th rule answer (by request-id hash) also goes to the teacher; it counts as a model call and becomes a labelled example. A rule whose spot-check precision drops below 0.8 after 5 checked spans is switched off for the rest of the stream.
Synthesis. Times are wall-clock seconds of the offline run on one CPU core. Re-running sim.py gives the same rule sets in the same order (checked twice, in separate processes).
Rules in your browser. engine.js is a JavaScript port of RuleChef's executor and of the gate. On all 3,400 sentences of this demo and all rule set versions it returns the same spans and gate decisions as the Python code.

Credits and licences

RuleChef (KRLabsOrg/rulechef, Apache-2.0): rule learning from examples. The search synthesis used here was installed from commit 8173f96 of its feat/search-synthesis branch.
vela3-fast (vllm-sr/vela3-fast, Apache-2.0): the teacher.
TAB, the Text Anonymization Benchmark (Pilán et al., 2022, Norsk Regnesentral, repository): MIT licence, copyright (C) 2021-2026 Norsk Regnesentral, full notice in DATA_LICENSE_TAB.txt. Used here: its public train split as published at mattmdjaga/text-anonymization-benchmark-train (revision f0b7eeb). The sentences, TAB's labels for the three types and the teacher's answers are in data/requests.jsonl.
This Space's code: Apache-2.0.