baseline-zeroshot-nt-esm
Refbaseline-zeroshot-nt-esm
Suite score
0.397
task headline
Suite score
0.43partial
1.0 = SOTA
Wall clock
—
of 24 h budget
Compute
H100:8
Agent cost
—
Started
—
Results by dataset
Raw metric, 95% CI where reported, and the normalized score against the dataset's SOTA (or reference).
| Dataset | Metric | Value | 95% CI | Normalized | Chance | Ref. | SOTA | n · coverage |
|---|---|---|---|---|---|---|---|---|
| clinvar gene holdout 2k Variant effect (DNA, clinical + GWAS) | AUPRC | 0.690 | — | 0.45 | 0.500 | 0.851 | 0.919 | 2,000 · 100% |
| songlab clinvar vs benign Variant effect (DNA, clinical + GWAS) | AUPRC | 0.678 | — | 0.52 | 0.434 | 0.793 | 0.906 | 5,000 · 100% |
| songlab cosmic Variant effect (DNA, clinical + GWAS) | AUPRC | 0.126 | — | 0.16 | 0.037 | 0.193 | 0.581 | 5,000 · 100% |
| songlab ukb finemapped coding Variant effect (DNA, clinical + GWAS) | AUPRC | 0.093 | — | -0.04 | 0.101 | 0.168 | 0.306 | 2,221 · 100% |
| traitgym mendelian Variant effect (DNA, clinical + GWAS) | chrom weighted auprc | 0.190 | — | 0.10 | 0.122 | 0.712 | 0.779 | 3,380 · 100% |
| traitgym complex Variant effect (DNA, clinical + GWAS) | chrom weighted auprc | 0.125 | — | -0.11 | 0.140 | 0.137 | 0.274 | 2,000 · 100% |
| caqtl african Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.102 | — | 0.03 | 0.085 | — | 0.576 | 5,000 · 100% |
| caqtl european Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.085 | — | 0.01 | 0.083 | — | 0.342 | 5,000 · 100% |
| dsqtl yoruba Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.018 | — | -0.00 | 0.020 | — | 0.631 | 8,226 · 100% |
| bqtl spi1 Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.031 | — | 0.01 | 0.028 | — | 0.600 | 5,000 · 100% |
| eqtl causality Molecular QTLs (chromatin, binding, expression, polyadenylation) | auroc per tissue mean | 0.462 | — | -0.18 | 0.500 | 0.670 | 0.717 | 2,000 · 100% |
| eqtl sign Molecular QTLs (chromatin, binding, expression, polyadenylation) | auroc per tissue mean | 0.513 | — | 0.04 | 0.500 | 0.750 | 0.834 | 2,000 · 100% |
| eqtl coefficient Molecular QTLs (chromatin, binding, expression, polyadenylation) | spearman per tissue mean | 0.015 | — | 0.03 | 0.000 | 0.390 | 0.511 | 2,000 · 100% |
| paqtl Molecular QTLs (chromatin, binding, expression, polyadenylation) | matched auprc | 0.619 | — | -0.38 | 0.628 | 0.621 | 0.653 | 1,343 · 100% |
| proteingym substitutions Protein fitness (DMS, protein input) | spearman per assay mean | 0.402 | — | 0.78 | 0.000 | 0.396 | 0.514 | 16,963 · 100% |
| rnagym ncrna Non-coding RNA fitness (DMS) | spearman per assay mean | 0.164 | — | 0.68 | 0.000 | 0.155 | 0.240 | 23,113 · 100% |
| rnagym mrna coding Coding-sequence fitness (DMS, nucleotide input) | spearman per assay mean | 0.129 | — | 0.51 | 0.000 | 0.175 | 0.251 | 14,000 · 100% |
| evee sge dms Coding-sequence fitness (DMS, nucleotide input) | spearman per assay mean | 0.122 | — | 0.25 | 0.000 | 0.355 | 0.492 | 6,308 · 100% |
| enformer tracks 128 Functional-genomics track prediction | track pearson | — | — | — | 0.000 | — | — | 1,000 · 100% |
| chrombpnet imr90 atac Functional-genomics track prediction | profile pearson | — | — | — | 0.000 | 0.750 | 0.860 | 12,920 · 100% |
| encode accessibility tiles Functional-genomics track prediction | track pearson | — | — | — | 0.000 | 0.817 | — | 1,024 · 100% |
| gene mteb hv1 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.738 | — | 0.73 | 0.500 | 0.721 | 0.828 | 15,404 · 100% |
| gene mteb hv2 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.704 | — | 0.84 | 0.500 | 0.652 | 0.742 | 14,956 · 100% |
| gene mteb hv3 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.735 | — | 0.70 | 0.500 | 0.758 | 0.835 | 15,744 · 100% |
| gene mteb hv4 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.703 | — | 0.80 | 0.500 | 0.697 | 0.755 | 15,633 · 100% |
| t7 gp17 recovery diagnostics | masked nt accuracy | 0.276 | — | 0.28 | 0.250 | — | — | 15 · 100% |
Variant effect (DNA, clinical + GWAS) 0.18Molecular QTLs (chromatin, binding, expression, polyadenylation) -0.01Protein fitness (DMS, protein input) 0.78Non-coding RNA fitness (DMS) 0.68Coding-sequence fitness (DMS, nucleotide input) 0.38Functional-genomics track prediction 0.00Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) 0.77
Integrity
Reference result.
Pending
- No checks recorded.
Run details
- Kind
- baseline
- Suite
- genomic_suite v1
- Started
- —
- Finished
- —
- Artifacts
- —