Build a genomic foundation model, scored on the full public radical-eval suite
dna_rna_protein_sequencetier T324 h budgetheadline: Suite scoresuite genomic_suite v1
About this task
The agent trains or post-trains sequence model(s) in 24 hours on 8×H100. Its final model is re-run by us, with no network, on ~25 public held-out datasets spanning DNA variant effect, chromatin/expression QTLs, protein, RNA and nucleotide deep mutational scans, genomic track prediction and sequence embeddings (the public part of the radical-eval suite). Every dataset is named in the prompt; labels are never shipped.
Best Suite score
0.101
Unknown agent
Best score vs SOTA
0.12
1.0 = SOTA on each dataset
Agent runs
1
2 reference results
Last run
—
26 core datasets · 0 private
Leaderboard
Ranked by suite score (normalized, 1.0 = SOTA). Dataset cells show normalized values; hover for raw values and CIs.
| # | Run | Integrity | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Unknown agentpilot-claude-40m-v2 | 0.101 | 0.12 | -0.03 | 0.01 | -0.00 | -0.03 | -0.02 | 0.14 | -0.00 | 0.02 | -0.00 | 0.01 | -0.09 | 0.02 | -0.13 | 0.04 | 0.78 | -0.08 | 0.06 | -0.04 | 0.00 | -0.00 | -0.00 | 0.00 | 0.01 | 0.00 | -0.00 | 0.27 | Pending | — | — |
| Refbaseline-io-smoke | 0.026 | 0.06 | 0.04 | -0.05 | 0.01 | 0.04 | 0.00 | 0.08 | -0.01 | -0.02 | 0.00 | -0.00 | 0.04 | -0.01 | -0.02 | 0.54 | 0.02 | 0.01 | 0.03 | 0.03 | -0.00 | -0.00 | -0.00 | 0.02 | 0.01 | 0.21 | 0.22 | 0.26 | — | — | ||
| Refbaseline-zeroshot-nt-esm | 0.397 | 0.43* | 0.45 | 0.52 | 0.16 | -0.04 | 0.10 | -0.11 | 0.03 | 0.01 | -0.00 | 0.01 | -0.18 | 0.04 | 0.03 | -0.38 | 0.78 | 0.68 | 0.51 | 0.25 | · | · | · | 0.73 | 0.84 | 0.70 | 0.80 | 0.28 | — | — |
* partial suite score (some core datasets not evaluated). ≈ approximate reference value. “inf.” = agent / effort inferred from the run id or trace (no run_meta.json).
Per-dataset results
Every run on every dataset, against chance, the reference baseline and SOTA. Filters above apply.
agent runbest runbaselineSOTAreferencechance95% CI privateRaw metric values; each row has its own axis.
Suite datasets
From suite.yaml. Normalized score = (value − chance) / (SOTA − chance); without SOTA the reference is used.
| Dataset | Visibility | Metric | Chance | Reference | SOTA | Source |
|---|---|---|---|---|---|---|
| Variant effect (DNA, clinical + GWAS) | ||||||
| clinvar gene holdout 2k | public | AUPRC | 0.500 | 0.851 Evo 2 7B | 0.919 CADD v1.7 | NCBI ClinVar GRCh38 VCF clinvar_20240301.vcf.gz (nearest weekly to the Evo2-paper 2024-02-28 release), >=2-star, P/LP vs B/LB, ref/alt <=64 bp; reference scores joined from HF goodarzilab/evo2-clinvar (Brixi et al. 2025) |
| songlab clinvar vs benign | public | AUPRC | 0.434 | 0.793 Evo2_7B | 0.906 GPN-Star-V100 | HF songlab/clinvar_vs_benign test.parquet (missense ClinVar P vs common benign; Ye, Benegas et al. 2025, GPN-Star) |
| songlab cosmic | public | AUPRC | 0.037 | 0.193 Evo2_7B | 0.581 GPN-Star-V100 | HF songlab/cosmic test.parquet (frequent COSMIC somatic missense vs gnomAD common missense) |
| songlab ukb finemapped coding | public | AUPRC | 0.101 | 0.168 Evo2_7B | 0.306 GPN-Star-M447 | HF songlab/ukb_finemapped_coding (UKB fine-mapped missense, PIP>0.9 vs PIP<0.01) |
| traitgym mendelian | public | chrom weighted auprc | 0.122 | 0.712 CADD (raw) | 0.779 GPN-Star-M447 | HF songlab/TraitGym mendelian_traits_matched_9 test (rev 1fde1955; Benegas, Eraslan, Song 2025, bioRxiv 10.1101/2025.02.11.637758) |
| traitgym complex | public | chrom weighted auprc | 0.140 | 0.137 Evo 2 7B (LLR) | 0.274 CADD (raw) | HF songlab/TraitGym complex_traits_matched_9 test (rev 1fde1955) |
| Molecular QTLs (chromatin, binding, expression, polyadenylation) | ||||||
| caqtl african | public | AUPRC | 0.085 | — | 0.576 AlphaGenome | AlphaGenome eval file storage.googleapis.com/alphagenome/evals/caqtl_african_variant_causality_human_predictions.feather (LCL caQTL, DeGorter 2023 via ChromBPNet benchmark; AG test chromosomes 3,6,9,12,16,18,19,21) |
| caqtl european | public | AUPRC | 0.083 | — | 0.342 AlphaGenome | AlphaGenome eval caqtl_european_variant_causality_human_predictions.feather (original eQTL Catalogue QTD100018, Zenodo 13848268) |
| dsqtl yoruba | public | AUPRC | 0.020 | — | 0.631 AlphaGenome | AlphaGenome eval dsqtl_yoruba_variant_causality_human_predictions.feather (Degner 2012 GEO GSE31388; hg19 IDs) |
| bqtl spi1 | public | AUPRC | 0.028 | — | 0.600 AlphaGenome | AlphaGenome eval bqtl_spi1_variant_causality_human_predictions.feather (Tehranchi 2016 PU.1 bQTL via ChromBPNet benchmark) |
| eqtl causality | public | auroc per tissue mean | 0.500 | 0.670 Borzoi | 0.717 AlphaGenome | AlphaGenome eval eqtl_variant_catalogue_causality_gene_balanced_human_predictions.feather (GTEx v8 via eQTL Catalogue SuSiE, distance-matched) |
| eqtl sign | public | auroc per tissue mean | 0.500 | 0.750 Borzoi | 0.834 AlphaGenome | AlphaGenome eval eqtl_variant_borzoi_sign_human_predictions.feather (GTEx v8 SuSiE PIP>0.9) |
| eqtl coefficient | public | spearman per tissue mean | 0.000 | 0.390 Borzoi | 0.511 AlphaGenome | AlphaGenome eval eqtl_variant_borzoi_coefficient_human_predictions.feather |
| paqtl | public | matched auprc | 0.628 | 0.621 Borzoi | 0.653 AlphaGenome | AlphaGenome eval paqtl_variant_causality_human_predictions.feather + PolyA_DB v3 (storage.googleapis.com/seqnn-share/helper/polyadb_human_v3.csv.gz) |
| Protein fitness (DMS, protein input) | ||||||
| proteingym substitutions | public | spearman per assay mean | 0.000 | 0.396 ESM2 650M | 0.514 VenusREM | ProteinGym v1.3 DMS_substitutions (HF OATML-Markslab/ProteinGym_v1; marks.hms.harvard.edu/proteingym/ProteinGym_v1.3); Notin et al. 2023 |
| Non-coding RNA fitness (DMS) | ||||||
| rnagym ncrna | public | spearman per assay mean | 0.000 | 0.155 RNA-FM | 0.240 RNAErnie | RNAGym fitness_prediction (HF Marks-lab/RNAgym; Arora et al. 2025) — Ribozyme + tRNA + Aptamer assays |
| Coding-sequence fitness (DMS, nucleotide input) | ||||||
| rnagym mrna coding | public | spearman per assay mean | 0.000 | 0.175 Evo 1 (131k) | 0.251 Evo 2 7B | RNAGym mRNA-coding assays (nucleotide CDS + codon-level mutants) — public substitute for the private Evo2-paper nucleotide DMS panels |
| evee sge dms | public | spearman per assay mean | 0.000 | 0.355 Evo 2 7B zero-shot loss | 0.492 AlphaMissense | MaveDB (CC0): BRCA1 urn:mavedb:00000097-a-2 (Findlay 2018), BRCA2 00001225-a-1 (HAP1 SGE, doi 10.1038/s41586-024-08388-8), LDLR 00001269-a-1 (Tabet 2025), TP53 00001213-a-1 (Funk 2025); EVEE construction (Pearce et al. 2026) |
| Functional-genomics track prediction | ||||||
| enformer tracks 128 | public | track pearson | 0.000 | — | — | Basenji2/Enformer human test set gs://basenji_barnyard/data/human (requester-pays), Avsec et al. 2021 |
| chrombpnet imr90 atac | public | profile pearson | 0.000 | 0.750 ChromBPNet (ENCODE predicted bigWig) | 0.860 AlphaGenome | ENCODE ChromBPNet annotation ENCSR978WIX (IMR-90 ATAC, experiment ENCSR200OML); fold-0 test peaks (chr1,3,6) from ENCFF802KJC; observed bigWig ENCFF650PHA |
| encode accessibility tiles | public | track pearson | 0.000 | 0.817 ChromBPNet (ENCODE predicted bigWigs) | — | ENCODE ChromBPNet annotations, observed 1 bp bigWigs: GM12878, K562, HepG2, A549, HCT116 x {ATAC, DNase} (e.g. DNase ENCSR003WJE/ENCSR296UHQ/ENCSR006CUK/ENCSR422EZD/ENCSR038BXQ; ATAC ENCSR389HIH/ENCSR467RSV/ENCSR380YGX/ENCSR311VBG/ENCSR799MEK) — public substitute for internal radical_track_pred / radical_catr / radical_dstr |
| Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | ||||||
| gene mteb hv1 | public | gene mteb 8shot | 0.500 | 0.721 NT-2.5b-1000g | 0.828 METAGENE-1 | HF metagene-ai/HumanVirusInfecting config class-1 (sha 70876feb; Liu et al. 2025, METAGENE-1, arXiv 2501.02045) |
| gene mteb hv2 | public | gene mteb 8shot | 0.500 | 0.652 NT-2.5b-Multi | 0.742 METAGENE-1 | HF metagene-ai/HumanVirusInfecting config class-2 |
| gene mteb hv3 | public | gene mteb 8shot | 0.500 | 0.758 NT-2.5b-Multi | 0.835 METAGENE-1 | HF metagene-ai/HumanVirusInfecting config class-3 |
| gene mteb hv4 | public | gene mteb 8shot | 0.500 | 0.697 METAGENE-1 | 0.755 NT-2.5b-1000g | HF metagene-ai/HumanVirusInfecting config class-4 |
| diagnostics | ||||||
| t7 gp17 recovery | public | masked nt accuracy | 0.250 | — | — | NCBI NC_001604.1 (T7 phage, 39,937 bp), gp17 tail fiber; 4 masked ~201 bp windows (radical-eval t7_design_recovery) |