Codex · high
Pending20260722T065459-codex-highharness: Codexmodel: unknowneffort: high(inferred)tag: codex-high
Mean AUPRC
0.655
task headline
Suite score
0.80
1.0 = SOTA
Wall clock
—
of 10 h budget
Compute
1xH100
Agent cost
—
Started
Jul 22, 2026
Results by dataset
Raw metric, 95% CI where reported, and the normalized score against the dataset's SOTA (or reference).
| Dataset | Metric | Value | 95% CI | Normalized | Chance | Ref. | SOTA | n · coverage |
|---|---|---|---|---|---|---|---|---|
| traitgym complex Variant effect (DNA) | AUPRC | 0.259 | — | 0.61 | 0.100 | 0.284 | 0.362 | 11,400 · 100% |
| traitgym mendelian Variant effect (DNA) | AUPRC | 0.712 | — | 0.79 | 0.101 | 0.648 | 0.875 | 3,353 · 100% |
| clinvar Variant effect (DNA) | AUPRC | 0.993 | — | 1.01 | 0.494 | — | 0.990 | 29,953 · 100% |
| clinvar missensesupp. Variant effect (DNA) | AUROC | 0.964 | — | 0.98 | 0.500 | 0.966 | 0.972 | 7,861 · 100% |
| clinvar intronicsupp. Variant effect (DNA) | AUROC | 0.921 | — | 0.87 | 0.500 | — | 0.984 | 4,368 · 100% |
| clinvar synonymoussupp. Variant effect (DNA) | AUROC | 0.962 | — | 1.00 | 0.500 | — | 0.961 | 8,155 · 100% |
| clinvar stop gainedsupp. Variant effect (DNA) | AUROC | 0.795 | — | 0.74 | 0.500 | — | 0.900 | 5,762 · 100% |
All reported metrics
| Slice | AUPRC | AUROC | Brier |
|---|---|---|---|
| traitgym complex | 0.259 | 0.652 | 0.085 |
| traitgym mendelian | 0.712 | 0.847 | 0.054 |
| clinvar | 0.993 | 0.993 | 0.029 |
| clinvar missense | 0.986 | 0.964 | 0.064 |
| clinvar intronic | 0.713 | 0.921 | 0.043 |
| clinvar synonymous | 0.660 | 0.962 | 0.009 |
| clinvar stop gained | 0.999 | 0.795 | 0.005 |
Breakdown: clinvar by consequence
| Slice | n | AUPRC | AUROC | Brier |
|---|---|---|---|---|
| 3 prime utr | 445 | 0.022 | 0.901 | 0.025 |
| 5 prime utr | 170 | 0.492 | 0.814 | 0.118 |
| initiator codon | 163 | 0.997 | 0.879 | 0.113 |
| intronic | 4,368 | 0.713 | 0.921 | 0.043 |
| missense | 7,861 | 0.986 | 0.964 | 0.064 |
| other | 133 | 0.560 | 0.855 | 0.057 |
| splice acceptor | 1,231 | 0.998 | 0.838 | 0.005 |
| splice donor | 1,649 | 1.000 | 0.844 | 0.004 |
| stop gained | 5,762 | 0.999 | 0.795 | 0.005 |
| synonymous | 8,155 | 0.660 | 0.962 | 0.009 |
Breakdown: clinvar by review status
| Slice | n | AUPRC | AUROC | Brier |
|---|---|---|---|---|
| 2 star | 28,058 | 0.993 | 0.993 | 0.029 |
| 3 star expert panel | 1,764 | 0.997 | 0.986 | 0.031 |
| unknown | 124 | 0.995 | 0.996 | 0.016 |
Breakdown: clinvar by significance pair
| Slice | n | AUPRC | AUROC | Brier |
|---|---|---|---|---|
| strong only P vs B | 20,145 | 0.995 | 0.992 | 0.030 |
| hedged only LP vs LB | 9,790 | 0.984 | 0.993 | 0.028 |
| any P vs any B | 29,935 | 0.993 | 0.993 | 0.029 |
Integrity
Post-run checks (docs/INTEGRITY.md). Runs publish once checks pass.
Pending
- pendingleakage auditNot individually covered by AUDIT_traitgym_mendelian.md
- missingtraining manifestNo training_manifest.json in the run folder
- missingtraceNo agent_log.jsonl kept for this run
- pendingoverlapAutomated overlap audit not yet run
Run details
- Kind
- agent
- Suite
- vep_multieval v1
- Started
- Jul 22, 2026 06:54 UTC
- Finished
- —
- Artifacts
- —
Agent trace
No trace was kept for this run.