Model Accuracy versus Cost (Vega-Lite)
Candidate serving configurations plotted on a log cost axis against exact-match accuracy, with the routed setup sitting on the frontier.
Rendering…
Make it your own.
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"title": "Evaluation frontier - accuracy against cost per 1,000 requests",
"data": {
"values": [
{"model": "Small 8B", "family": "Open weights", "cost": 0.42, "accuracy": 0.71},
{"model": "Small 8B + rerank", "family": "Open weights", "cost": 0.61, "accuracy": 0.78},
{"model": "Fine-tuned 8B", "family": "Open weights", "cost": 0.55, "accuracy": 0.83},
{"model": "Mid 70B", "family": "Open weights", "cost": 2.10, "accuracy": 0.86},
{"model": "Frontier", "family": "Hosted", "cost": 9.40, "accuracy": 0.93},
{"model": "Frontier mini", "family": "Hosted", "cost": 1.30, "accuracy": 0.85},
{"model": "Router (mixed)", "family": "Hybrid", "cost": 1.05, "accuracy": 0.90}
]
},
"encoding": {
"x": {"field": "cost", "type": "quantitative", "title": "USD per 1,000 requests", "scale": {"type": "log"}},
"y": {"field": "accuracy", "type": "quantitative", "title": "Exact-match accuracy", "scale": {"domain": [0.65, 1.0]}}
},
"layer": [
{
"mark": {"type": "point", "filled": true, "size": 150},
"encoding": {"color": {"field": "family", "type": "nominal", "title": "Family"}}
},
{
"mark": {"type": "text", "dy": -14, "fontSize": 10},
"encoding": {"text": {"field": "model", "type": "nominal"}}
}
]
}