baseline-io-smoke
Refbaseline-io-smoke
Suite score
0.026
task headline
Suite score
0.06
1.0 = SOTA
Wall clock
—
of 24 h budget
Compute
H100:8
Agent cost
—
Started
—
Results by dataset
Raw metric, 95% CI where reported, and the normalized score against the dataset's SOTA (or reference).
| Dataset | Metric | Value | 95% CI | Normalized | Chance | Ref. | SOTA | n · coverage |
|---|---|---|---|---|---|---|---|---|
| clinvar gene holdout 2k Variant effect (DNA, clinical + GWAS) | AUPRC | 0.515 | — | 0.04 | 0.500 | 0.851 | 0.919 | 2,000 · 100% |
| songlab clinvar vs benign Variant effect (DNA, clinical + GWAS) | AUPRC | 0.412 | — | -0.05 | 0.434 | 0.793 | 0.906 | 5,000 · 100% |
| songlab cosmic Variant effect (DNA, clinical + GWAS) | AUPRC | 0.040 | — | 0.01 | 0.037 | 0.193 | 0.581 | 5,000 · 100% |
| songlab ukb finemapped coding Variant effect (DNA, clinical + GWAS) | AUPRC | 0.109 | — | 0.04 | 0.101 | 0.168 | 0.306 | 2,221 · 100% |
| traitgym mendelian Variant effect (DNA, clinical + GWAS) | chrom weighted auprc | 0.123 | — | 0.00 | 0.122 | 0.712 | 0.779 | 3,380 · 100% |
| traitgym complex Variant effect (DNA, clinical + GWAS) | chrom weighted auprc | 0.151 | — | 0.08 | 0.140 | 0.137 | 0.274 | 2,000 · 100% |
| caqtl african Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.082 | — | -0.01 | 0.085 | — | 0.576 | 5,000 · 100% |
| caqtl european Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.077 | — | -0.02 | 0.083 | — | 0.342 | 5,000 · 100% |
| dsqtl yoruba Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.021 | — | 0.00 | 0.020 | — | 0.631 | 8,226 · 100% |
| bqtl spi1 Molecular QTLs (chromatin, binding, expression, polyadenylation) | AUPRC | 0.028 | — | -0.00 | 0.028 | — | 0.600 | 5,000 · 100% |
| eqtl causality Molecular QTLs (chromatin, binding, expression, polyadenylation) | auroc per tissue mean | 0.510 | — | 0.04 | 0.500 | 0.670 | 0.717 | 2,000 · 100% |
| eqtl sign Molecular QTLs (chromatin, binding, expression, polyadenylation) | auroc per tissue mean | 0.497 | — | -0.01 | 0.500 | 0.750 | 0.834 | 2,000 · 100% |
| eqtl coefficient Molecular QTLs (chromatin, binding, expression, polyadenylation) | spearman per tissue mean | -0.009 | — | -0.02 | 0.000 | 0.390 | 0.511 | 2,000 · 100% |
| paqtl Molecular QTLs (chromatin, binding, expression, polyadenylation) | matched auprc | 0.642 | — | 0.54 | 0.628 | 0.621 | 0.653 | 1,343 · 100% |
| proteingym substitutions Protein fitness (DMS, protein input) | spearman per assay mean | 0.009 | — | 0.02 | 0.000 | 0.396 | 0.514 | 16,963 · 100% |
| rnagym ncrna Non-coding RNA fitness (DMS) | spearman per assay mean | 0.002 | — | 0.01 | 0.000 | 0.155 | 0.240 | 23,113 · 100% |
| rnagym mrna coding Coding-sequence fitness (DMS, nucleotide input) | spearman per assay mean | 0.006 | — | 0.03 | 0.000 | 0.175 | 0.251 | 14,000 · 100% |
| evee sge dms Coding-sequence fitness (DMS, nucleotide input) | spearman per assay mean | 0.013 | — | 0.03 | 0.000 | 0.355 | 0.492 | 6,308 · 100% |
| enformer tracks 128 Functional-genomics track prediction | track pearson | -0.000 | — | -0.00 | 0.000 | — | — | 1,000 · 100% |
| chrombpnet imr90 atac Functional-genomics track prediction | profile pearson | -0.000 | — | -0.00 | 0.000 | 0.750 | 0.860 | 12,920 · 100% |
| encode accessibility tiles Functional-genomics track prediction | track pearson | -0.001 | — | -0.00 | 0.000 | 0.817 | — | 1,024 · 100% |
| gene mteb hv1 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.507 | — | 0.02 | 0.500 | 0.721 | 0.828 | 15,404 · 100% |
| gene mteb hv2 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.504 | — | 0.01 | 0.500 | 0.652 | 0.742 | 14,956 · 100% |
| gene mteb hv3 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.571 | — | 0.21 | 0.500 | 0.758 | 0.835 | 15,744 · 100% |
| gene mteb hv4 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) | gene mteb 8shot | 0.556 | — | 0.22 | 0.500 | 0.697 | 0.755 | 15,633 · 100% |
| t7 gp17 recovery diagnostics | masked nt accuracy | 0.257 | — | 0.26 | 0.250 | — | — | 15 · 100% |
Variant effect (DNA, clinical + GWAS) 0.02Molecular QTLs (chromatin, binding, expression, polyadenylation) -0.00Protein fitness (DMS, protein input) 0.02Non-coding RNA fitness (DMS) 0.01Coding-sequence fitness (DMS, nucleotide input) 0.03Functional-genomics track prediction -0.00Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) 0.12
Integrity
Reference result.
Pending
- No checks recorded.
Run details
- Kind
- baseline
- Suite
- genomic_suite v1
- Started
- —
- Finished
- —
- Artifacts
- —