Which decision model works on your data?
Public benchmarks rarely match your task. Paste or upload labeled examples, run them through up to 8 decision models at once, and get accuracy, calibration, latency and cost side by side — plus the model and confidence threshold to ship, and the code to call it.
How it works
- 1
Bring labeled examples
Paste rows straight from a spreadsheet, upload CSV or JSONL, or start from a sample dataset. You need a text column and a label column.
- 2
Define each label
Each distinct label becomes an option. One plain-English line per option is all the model sees — no training.
- 3
Pick models and run
See the credit estimate before you start. Each model runs in parallel; failed rows are refunded.
- 4
Ship the winner
Get the recommended model and threshold for your target error rate, the rows models get wrong, and a ready-to-run API call.
What you can run
- Without an account: up to 20 rows × 2 models, once a day.
- Your first evaluation after signing up is free: up to 100 rows × 4 models.
- After that, up to 500 rows × all 8 models, paid in credits — 200 rows on every model costs about 2,200 credits.
Example reports
Already run on public datasets, so you can see a full report before bringing your own data.
Run an evaluation
1. Data
FAQ
- What is a decision model eval?
- A decision model eval runs the same labeled examples through several decision models — zero-shot classifiers that return a calibrated probability for each option — and compares how often each one is right, how well its confidence matches reality, how fast it is and what it costs.
- How many examples do I need?
- Accuracy is shown with a 95% confidence interval, so you can see how much it would move with more data. Threshold recommendations need at least 50 answered rows; 100–300 rows covering every label is a good start.
- Which data is sent where?
- Each row goes to the providers of the models you select: TypeSafe (Jev), Cloudflare (Clef), and through OpenRouter to Perplexity, OpenAI, Microsoft and Liquid AI. Untick a model to keep data away from its provider. Runs without an account are not stored; signed-in runs are kept for 30 days.
- Is a decision model just a zero-shot classifier?
- Largely, yes: you describe the options in plain English and get a probability for each, without training. What matters in production is whether those probabilities are calibrated on your data and which threshold keeps errors acceptable — which is what this eval measures.
- How is the recommended threshold chosen?
- For each model we scan confidence thresholds from 0.50 to 0.99 and keep the one that covers the most rows while the error rate among covered rows stays at or below your target (1%, 5% or 10%). The recommended model is the one with the highest coverage.