We didn't want a vendor. So we built the machine.
Expert data at this standard cannot be produced with a spreadsheet, a Slack channel, and a contractor pool. The verification has to be structural. So the platform came first, and the data is what comes out of it.
Five systems, one chain of custody
Each stage hands the next a signed record. Nothing enters a dataset without passing through all five, and the manifest that ships with your data is that record.
Expert authoring
Practitioners work in an environment that mirrors their real tools rather than a labelling UI. The workbench captures the intermediate state — the revision, the abandoned branch, the formula before it was corrected — because that is the part worth having.
- Spreadsheet, repo, notebook and document surfaces
- Step-level capture, not just final artefacts
- Task briefs with inline acceptance criteria
- Draft, submit, and revise-on-rejection flow
Sandboxed verification
Graders are ordinary Python, executed in an isolated sandbox with no network by default. They run on expert submissions and on model rollouts through the same path, so the reward a model trains against is the reward a human was held to.
- Deterministic replay and snapshot restore
- Seed sweeps with flake detection
- Partial-credit rubric primitives
- Versioned graders pinned to task specs
Review that escalates
Every submission goes to a second independent expert. Where the grader and the reviewer disagree, the item routes to a senior adjudicator rather than being resolved by majority or quietly dropped.
- Blind second review
- Disagreement routing and audit trail
- Reviewer calibration scoring
- Structured rejection reasons returned to the author
Provenance & versioning
An append-only record of every task, author credential class, grader version, verdict and exposure event. Datasets are immutable and versioned; a benchmark score is reproducible against exactly what produced it.
- Immutable dataset versions
- Exposure tracking for contamination hygiene
- Rights assignment recorded per contributor
- Manifest exported with every delivery
It lands where your training run already looks
The last system is the boring one that matters: getting verified work into your pipeline in your shape, on a schedule, without a human copying files.
# a delivered record, abridged
{
"task_id": "LYD-4471",
"domain": "finance.lbo",
"spec_version": "3",
"author": { "credential_class": "vp_sponsors_coverage", "verified": true },
"artifacts": [ "model.xlsx", "trace.jsonl", "rationale.md" ],
"grader": { "id": "cell_diff", "version": "1.4.2", "seeds": 200 },
"verdict": "verified",
"review": { "second_reviewer": true, "adjudicated": false },
"exposure": [ ],
"rights": "assigned"
}
Run it with us, or run it yourself
Some teams want the data. Some teams have their own experts — clinicians on staff, engineers in-house, an analyst bench — and want the system that turns them into training signal.
We run the platform and the network
You write the spec and set the acceptance criteria. We recruit, commission, verify and deliver. The platform is our problem.
- You provide
- A capability target and a definition of done
- We provide
- Experts, task design, graders, review, delivery
- Best for
- Teams who need a domain they don't have in-house
The platform in your own cloud
Deployed into your VPC for teams whose data can't leave it. Bring your own experts, or run a hybrid with ours. Your infrastructure, your access controls, your audit logs.
- You provide
- Your cloud account and your practitioners
- We provide
- The workbench, runtime, review system and support
- Best for
- Regulated data, or in-house expertise you already pay for
Built for data that can't leak
Our contributors are practising professionals with employers and obligations. That constrains the architecture, and it should.
- Isolation
- Grader execution is sandboxed with no outbound network by default. Environments are ephemeral and destroyed after capture.
- Least privilege
- Contributors see only the task they are commissioned for. Client identity is not disclosed to the network unless a client asks for it to be.
- Confidentiality
- Every contributor is under NDA before they receive a brief, and tasks are constructed so that no contributor is asked to reproduce their employer's proprietary material.
- Residency
- Managed workloads run in a region you nominate. Deployed installations never send task content to us at all.
See the platform run
A walkthrough is a working session, not a slide deck: a real task, a real grader, and a rejection.