Open source · Apache-2.0 · pre-alpha

Keep your decision thresholds true.

trimset is decision ops for decision models like Jev, Ollaya and Ollama. It logs every decision, joins it to what really happened, refits your thresholds and tells you when the model drifts.

Get started Star on GitHub

$ docker compose up -d # trimset on :8800

One line to adopt

# before: straight to your model
base_url = "https://api.typesafe.ai"

# after: through trimset, zero code changes
base_url = "https://trimset.example.com/p/tsp_…"

# then gate with the live policy, locally
d = ts.gate("refund_fraud", p=0.83, model_served="jev-1.14.0")
d.action  # ESCALATE  (policy v7)

Works with any /v1/systemone server Jev Ollaya Ollama OpenAI decisions LangGraph decision-circuits All integrations →

Thresholds you set once go stale.

Everyone writes if p > 0.9 and never checks it again. trimset closes the loop in four steps.

01 · Capture

Log every decision

Through the proxy or the logging API. Probabilities, model version, latency and cost. Raw state is never stored.

02 · Join

Match real outcomes

Chargebacks, reviewer labels, reassignments. Outcomes arrive hours or months later and attach to the original decision.

03 · Refit

Compile a policy

State a target like "at most 2% false passes". trimset recalibrates and proposes thresholds and an escalation band.

04 · Enforce

Ship it to your apps

You approve, the SDK picks it up and gates locally, with no network call on the request path. Roll back with one command.

How it works, step by step →

Every threshold is a business trade-off. See it.

Pick how much manual review you can afford per error avoided. trimset shows the curve on your own traffic, not a vendor benchmark.

refund_fraud · one run of the M2 demo on generated data · errors counted on 1,127 refunds the fit never saw
Accept below	Automated	Auto-path errors	Sent to review
0.10	91.9%	0.58%	8.1%
0.26 ✓ proposed	96.3%	1.66%	3.7%
0.30	97.3%	2.19%	2.7%

Six decisions, one loop

Each use case is a real question an app asks a decision model, the outcome that comes back later, and the target its policy has to meet.

All use cases →

noulDemo available Refund fraud Is this refund request fraudulent? Chargebacks arrive 30 to 90 days later. choiceDemo available Support ticket triage Which team should handle this ticket? The team that closes it says, hours later. scoreComing soon Model routing How hard is this request? Small model or large model, priced per call. noulComing soon PII guardrail Does this text contain personal data? Checked before a prompt reaches an LLM. noulComing soon Agent tool approval Is this tool call safe to run without a human? Unsafe passes stay rare. scoreComing soon Content moderation How serious is the problem with this listing? The review queue has a daily cap.

Built for System One models

Pre-alpha. The roadmap says what works today and what is coming.

Calibration per question

Temperature, Platt and isotonic fits per question wording and model. Never pooled across them.

Escalation bands

Confident cases run on System One. The uncertain middle goes to an LLM or a human.

Drift alerts that mean it Coming soon

Label-free distribution drift with significance tests, so small samples don't page you at 3 a.m.

Silent upgrade alerts Coming soon

Know the moment an alias like jev-latest starts answering with a new version. trimset already records which version answered.

Shadow models Coming soon

Run an open model next to your hosted one on live traffic and compare before you switch.

Review queue Coming soon

Escalated cases land in a queue. Every reviewer label becomes training signal for the next refit.

Self-hosted. Your data stays yours.

One docker compose, Postgres underneath, no account needed. A Python SDK; TypeScript comes in v0.2.

Read the docs Self-hosting

git clone https://github.com/trimset/trimset
cd trimset
# two secrets in .env first: see Self-hosting
docker compose up -d

pip install trimset        # the Python SDK, from the first release