OneTriangle
Contact

Fine-tuned Mapper API

Big-model answers. Small-model prefill. Trained on your task.

A small model reads your document, a trained mapper converts its KV cache into a big model's representation, and the big model answers without ever reading the document. You send one JSONL file of questions over your documents; we train, benchmark, and host the mapper, or hand you the artifact.

Request an API key See all benchmarks

Measured results

Production pair: Qwen3.5-4B (reader) → Qwen3.8-27B (answerer), n=100 held-out questions per benchmark, 95% document-grouped CIs. Your evaluation report runs the same four arms on your own held-out questions.

Benchmark4B aloneFine-tuned mapper27B native
QuALITY (long stories)0.8500.9000.900
BAMBOO PaperQA (papers)0.8570.8810.869
MATH-5000.9300.9000.960
QMSum (meetings, judge)0.4400.4240.497

The pattern holds across pairs: on Qwen3 1.7B→8B, a task-specific mapper trained on 465 documents matches native 8B on ZebraLogic exactly (0.79 vs 0.79) at about 39% of the 8B's TTFT at 32K context, and beats a parameter-matched LoRA of the small model on the same data. Mappers pay off most on long, domain-specific documents: contracts, papers, patient records, codebases. Short-context tasks (see MATH-500) gain little, because the small model is already close.

How it works

  1. Send data. One JSONL file: questions over your documents. No GPUs, no model weights, no configs on your side.
  2. We train. The two models read your training documents, the big model answers your training questions itself, and a rank-16 adapter is distilled to those answers (2–6 hours). Your gold labels are used only for your benchmark report, never for training.
  3. You verify, then ship. The evaluation report scores all four arms on 100 held-out questions whose documents were never trained on. Then call hosted inference, or take the packaged artifact (safetensors + hash manifest) into your own stack.

Quick start

1 · Format your data (data.jsonl, one JSON object per line)

{"id": "q-0017", "group_id": "contract-07",
 "context": "<full document text>",
 "query": "Can the supplier terminate without cause? Answer Yes or No.",
 "reference": "No"}

group_id ties every question to its document; held-out splits are document-disjoint. Recommended: ≥1,000 questions over ≥50 documents (minimum 150 questions). Documents up to ~32k tokens.

2 · Create, train, evaluate

export KVT_API_KEY=kvt_sk_...
BASE=https://api.onetriangle.ai/v1

curl -X POST $BASE/projects -H "Authorization: Bearer $KVT_API_KEY" \
  -d '{"project": "support-bot", "data_path": "/uploads/data.jsonl"}'

curl -X POST $BASE/projects/support-bot/prepare -d '{"test_size": 100}' ...
curl -X POST $BASE/projects/support-bot/train ...        # poll until "trained"
curl -X POST $BASE/projects/support-bot/evaluate -d '{"n": 100}' ...

3 · Use it

# Hosted inference
curl -X POST $BASE/projects/support-bot/generate \
  -H "Authorization: Bearer $KVT_API_KEY" \
  -d '{"context": "<document>", "query": "<question>", "max_new": 512}'
# -> {"answer": "..."}

# Or take the artifact into your own stack
curl -X POST $BASE/projects/support-bot/package ...
# -> mapper.safetensors + SHA-256 manifest + configs

Acceptance, plainly

We suggest shipping when your report shows the mapper within 0.03 of the big model natively, or when the "gap closed" confidence interval excludes 0.5. If your report shows the mapper at small-model level, your contexts are probably too short to matter or there's too little training data; the report tells you which.

Request an API key