Task Router
A 322M-parameter classifier that makes ten routing decisions in one forward pass, served as an API and running in production inside PugPort.
Code: Waytrine/task-router, MIT. Weights: Hugging Face, Apache-2.0.
97.8%
Held-out synthetic test, v4. Untuned: 44.0%.
91.0%
Sealed real traffic, v5. v4, in production: 72.0%.
37ms
Median, short request
10
Heads, one forward pass
322M
Parameters
- Encoder
- mmBERT-base, ModernBERT, 22 layers
- License
- MIT code, Apache-2.0 weights
- Weights
- Hugging Face, v4
- Training data
- Chinese and English
- Input
- Up to 1024 tokens
- Status
- v4 in production; v5 evaluated, rolling out next
In production: PugPort
PugPort is the agent side: it runs many AI agents and accounts from one chat. Task Router is the model side: it reads each task and judges its type and difficulty, so PugPort can pick a cheaper model or a stronger one.
Ten decisions, one pass
Task Router reads a bounded state of five parts: the message, recent context, quoted reference text, attachment metadata (kind, name and type, never the contents) and the candidate executors. It answers ten single-choice questions in one forward pass.
PugPort acts on the answers. difficulty picks the model tier and the effort
level, task_type picks the routing category, and project_access,
run_commands, browser_use and media_output decide
whether a message gets a reply or becomes a task.
| Head | Answers |
|---|---|
| task_type | 10conversation, coding, research, writing, translation, data_analysis, media, automation, mixed, unclear |
| difficulty | 4simple, standard, complex, unclear |
| web_search | 3yes, no, unclear |
| document_read | 3yes, no, unclear |
| run_commands | 3yes, no, unclear |
| browser_use | 3yes, no, unclear |
| input_modality | 6text, image, audio, video, mixed, unclear |
| media_output | 9none, image_create, image_edit, speech, music, video_generate, video_render, mixed, unclear |
| project_access | 4none, read, write, unclear |
| executor_intent | 6none, assign, exclude, reassign, mention_only, unclear |
Four rounds, one day
Four fine-tuning rounds on 26 October 2026, each on more data. Held-out accuracy went from 44.0% untuned to 97.8%.
The synthetic dataset has 200 scenarios, each written in Chinese and English: 400 base rows in 169 families, written by an AI assistant and reviewed by agents.
Splits are grouped by family, with zero family overlap across train (240), validation (60), calibration (40) and test (60). A test row never has a sibling in training.
A second check, on an independent 24-item set frozen before inference: v4 scored 93.3%, 224 of 240 decisions.
| Model | Train rows | Mean | Exact |
|---|---|---|---|
| Untuned base | – | 44.0% | 0.0% |
| v1 | 240 | 79.8% | 21.7% |
| v2 | 400 | 83.7% | 31.7% |
| v3 | 520 | 96.2% | 70.0% |
| v4 | 720 | 97.8% | 78.3% |
Soft cross-entropy · 4 epochs · batch 8 × gradient accumulation 2 · learning rate 2.5e-5 encoder, 1e-4 heads · weight decay 0.01 · gradient clip 1.0 · full encoder fine-tune · bf16
97.8% on paper. 72.0% on real traffic.
A high score on synthetic data is a claim about synthetic data. I built a sealed test from real traffic to check it, found the gap, and closed most of it.
-
First, a test that can't leak
392 real requests from my own use, in 86 task groups, labelled blind by agents that never saw the model's prediction, with 69 labels cross-checked.
One item per group went into a sealed holdout of 30, fixed by a SHA-256 manifest. The 132 other records from those groups were quarantined, so nothing similar reached training.
-
Then, the cause per head
v4 scored 72.0% over 300 decisions. Seven heads scored 70% or more; three did not: difficulty 33.3%, project_access 26.7%, run_commands 40.0%.
The synthetic data under-represented long coding work that needs write access and commands, so the model rated most real work “standard”. On 154 historical production judgments, its difficulty grade agreed with reviewers 24.0% of the time.
-
Finally, 75 real rows
v5 retrained with 75 reviewed real rows, with 35 more held for development. Three variants scored 89.3%, 90.3% and 91.0% on the same sealed 30.
The best lifted difficulty to 80.0%, project_access and run_commands to 96.7%. The synthetic regression sets did not drop: 98.0%, 99.0%, 99.5%.
v4 72.0%
v5, trained from the base model 91.0%
Within 3 points of a model 12× its size
Same schema, same sealed 30 items. Task Router v5 lands within 3 points of models 2.5× and 12× its size.
| Model | Size | Setup | Accuracy |
|---|---|---|---|
| H2O-Lightning-4B | 4B | LoRA | 94.0% |
| Intern-Decision-0.8B | 0.8B | Fine-tuned | 93.0% |
| Task Router v5 | 322M | Fine-tuned | 91.0% |
| elephant-1 | 9B | Zero-shot | 90.3% |
| GLiNER2.5-multi | 287M | Fine-tuned | 83.0% |
The API: bounded, pinned, forgetful
POST /v1/judge and GET /healthz. The app never waits past its
deadline, and the server never queues more than eight jobs.
-
Client
A deadline, then rule-based routing. A 30 s back-off after an error. Pins the model version and the weight SHA-256.
-
Edge
Cloudflare Tunnel to a server bound to loopback. Per-IP rate limit.
-
Server
A raw ASGI app under Uvicorn, one worker.
-
Queue
One inference at a time, 8 waiting. Full: 503 with Retry-After. Waited over 2.5 s: dropped.
-
Model
Apple silicon, MPS, FP32. A warm-up prediction after 60 s idle. Every response carries the version and weight hash.
37ms
Median, short request (Apple silicon, MPS)
68ms
Median, medium request (Apple silicon, MPS)
140ms
Median, long request (Apple silicon, MPS)
82ms
Warm median, RTX 4060 laptop, p95 107 ms
Nothing kept that isn't needed
Logs keep time, status, latency, token count and country. Request bodies are not kept. Before anything leaves the app, credentials are redacted by pattern (private keys, auth headers, common provider API keys, JWTs, bearer tokens, URL credentials) and file paths are removed.
Sharing samples for training is optional: on by default, off by default in the EU/EEA, the UK and Switzerland, under a random install ID with a delete token whose hash is all the server stores.
Advice, never authority
The model's advice can never widen permissions. Explicit user pins, approved rules, quotas, budgets and read-only limits always win. When the advice conflicts with them, the conflict is recorded and the advice dropped.
Next
Roll out v5, trained and evaluated. Then run inference on the device itself.
Where it runs
It routes requests in PugPort, a multi-agent desktop app. Code: Waytrine/task-router (MIT), under the Waytrine organization. The v4 weights are on Hugging Face (Apache-2.0).