Developed at CaliperAI · VLA training data · autonomous driving
Annotation that reasons first — then sees the future.
CaliperGT turns driving scenes into chain-of-causation training samples: grounded reasoning, committed predictions, executable decisions — verified against what actually happened.
01 · What is a VLA?
See → reason → act, in one model
A vision-language-action model takes the sensor view in, reasons about it in language, and outputs the driving action.
The newest generation — reasoning VLAs like NVIDIA's Alpamayo-R1 — train on exactly this chain. The data has to contain the chain, not just the boxes.
02 · Why now
The need of the hour
Driving is becoming one model
Hand-built perception → prediction → planning pipelines are giving way to end-to-end models — Alpamayo-R1, EMMA, LINGO. Language is becoming the interface to driving.
Safety cases need the “why”
An end-to-end model that can't explain a decision can't be audited. Reasoning VLAs make every action legible — it arrives with its cause. That's what regulators and validation teams can work with.
The data doesn't exist yet
Billions of labeled frames; almost no causally sound reasoning traces. The long tail won't yield to more miles — it yields to models that reason. Producing that data is what CaliperGT is for.
03 · What VLA annotation means
One decision = one sample
At each decision point, the annotator produces four linked layers — on the same scene:
The catch: hindsight poisons reasoning
An annotator who already watched the ending writes rationalizations, not reasoning. The fix is structural:
04 · The platform
A guided flow in the order a driver thinks
One step per screen. Decision points arrive pre-mined from ego kinematics; every field autosaves into a versioned record.
Every commitment gets scored
Because everything is committed under the lock, each sample carries comparisons hindsight data can't contain:
Agreements train the policy. Divergences become counterfactual pairs. Verdicts train the judge.
05 · Built in today
What the platform provides
Decision-point mining
Changepoints over ego kinematics + map, placed at the earliest causal precursor.
Audited causal lock
Future frames hidden; every reveal recorded — who, when, and what was committed first.
Narrative guided flow
“I see… if ego does nothing… ego should… because…” — one step per screen; expert form one toggle away.
Category-aware actors
VRU / vehicle / traffic control auto-classified; attributes pre-fill from the annotator's own sentence.
No pre-labels required
Click-to-ground when tracks exist; phrase-to-actor when they don’t. Never blocked on perception.
Live QA + provenance
Completeness checklist as you type; source, editor and version trail on every record; taxonomy-driven vocabularies.
"sample": {
"observation": "VRUs crossing at a junction, red light",
"entities": ["VRUs↦track", "light↦red_for_ego"],
"prediction": { "VRUs": "cross · likely · 4s" },
"risk": { "level": "critical", "factors": ["vru", "rule_violation"] },
"action": { "recommended": "stop_for_constraint+keep_lane", "target_speed": 0 },
"verification": { "VRUs": "as_predicted", "risk": "averted_by_ego",
"future_revealed": "after_commitment" }
}06 · Next
Roadmap
AI pre-fill, human verify
A model drafts under the same lock; annotators correct. Corrections double as preference data.
Training-ready CoC export
Samples packaged for VLA fine-tuning and judge evaluation.
Trajectory sketching
Predicted actor paths and planned ego paths drawn on the BEV.
Map-element grounding
Mentions and action targets bound to lane / stop-line / signal ids.
Building data for reasoning-grade autonomy?
Tell us about your sensors, your models, and the decisions you need explained — we'll show you VLA grounding on your own scenes.