Unknown agent

Pending

pilot-claude-40m-v2harness: Unknown agentmodel: unknowneffort: default

Suite score

0.101

task headline

Suite score

0.12

1.0 = SOTA

Wall clock

—

of 24 h budget

Compute

H100:8

Agent cost

—

Started

—

Results by dataset

Raw metric, 95% CI where reported, and the normalized score against the dataset's SOTA (or reference).

Dataset	Metric	Value	95% CI	Normalized	Chance	Ref.	SOTA	n · coverage
clinvar gene holdout 2k Variant effect (DNA, clinical + GWAS)	AUPRC	0.487	—	-0.03	0.500	0.851	0.919	2,000 · 100%
songlab clinvar vs benign Variant effect (DNA, clinical + GWAS)	AUPRC	0.441	—	0.01	0.434	0.793	0.906	5,000 · 100%
songlab cosmic Variant effect (DNA, clinical + GWAS)	AUPRC	0.035	—	-0.00	0.037	0.193	0.581	5,000 · 100%
songlab ukb finemapped coding Variant effect (DNA, clinical + GWAS)	AUPRC	0.095	—	-0.03	0.101	0.168	0.306	2,221 · 100%
traitgym mendelian Variant effect (DNA, clinical + GWAS)	chrom weighted auprc	0.112	—	-0.02	0.122	0.712	0.779	3,380 · 100%
traitgym complex Variant effect (DNA, clinical + GWAS)	chrom weighted auprc	0.158	—	0.14	0.140	0.137	0.274	2,000 · 100%
caqtl african Molecular QTLs (chromatin, binding, expression, polyadenylation)	AUPRC	0.083	—	-0.00	0.085	—	0.576	5,000 · 100%
caqtl european Molecular QTLs (chromatin, binding, expression, polyadenylation)	AUPRC	0.088	—	0.02	0.083	—	0.342	5,000 · 100%
dsqtl yoruba Molecular QTLs (chromatin, binding, expression, polyadenylation)	AUPRC	0.020	—	-0.00	0.020	—	0.631	8,226 · 100%
bqtl spi1 Molecular QTLs (chromatin, binding, expression, polyadenylation)	AUPRC	0.034	—	0.01	0.028	—	0.600	5,000 · 100%
eqtl causality Molecular QTLs (chromatin, binding, expression, polyadenylation)	auroc per tissue mean	0.481	—	-0.09	0.500	0.670	0.717	2,000 · 100%
eqtl sign Molecular QTLs (chromatin, binding, expression, polyadenylation)	auroc per tissue mean	0.508	—	0.02	0.500	0.750	0.834	2,000 · 100%
eqtl coefficient Molecular QTLs (chromatin, binding, expression, polyadenylation)	spearman per tissue mean	-0.066	—	-0.13	0.000	0.390	0.511	2,000 · 100%
paqtl Molecular QTLs (chromatin, binding, expression, polyadenylation)	matched auprc	0.629	—	0.04	0.628	0.621	0.653	1,343 · 100%
proteingym substitutions Protein fitness (DMS, protein input)	spearman per assay mean	0.402	—	0.78	0.000	0.396	0.514	16,963 · 100%
rnagym ncrna Non-coding RNA fitness (DMS)	spearman per assay mean	-0.018	—	-0.08	0.000	0.155	0.240	23,113 · 100%
rnagym mrna coding Coding-sequence fitness (DMS, nucleotide input)	spearman per assay mean	0.015	—	0.06	0.000	0.175	0.251	14,000 · 100%
evee sge dms Coding-sequence fitness (DMS, nucleotide input)	spearman per assay mean	-0.018	—	-0.04	0.000	0.355	0.492	6,308 · 100%
enformer tracks 128 Functional-genomics track prediction	track pearson	0.000	—	0.00	0.000	—	—	1,000 · 100%
chrombpnet imr90 atac Functional-genomics track prediction	profile pearson	-0.001	—	-0.00	0.000	0.750	0.860	12,920 · 100%
encode accessibility tiles Functional-genomics track prediction	track pearson	-0.000	—	-0.00	0.000	0.817	—	1,024 · 100%
gene mteb hv1 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator)	gene mteb 8shot	0.500	—	0.00	0.500	0.721	0.828	15,404 · 100%
gene mteb hv2 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator)	gene mteb 8shot	0.503	—	0.01	0.500	0.652	0.742	14,956 · 100%
gene mteb hv3 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator)	gene mteb 8shot	0.501	—	0.00	0.500	0.758	0.835	15,744 · 100%
gene mteb hv4 Sequence embeddings (Gene-MTEB, probe fitted by the evaluator)	gene mteb 8shot	0.499	—	-0.00	0.500	0.697	0.755	15,633 · 100%
t7 gp17 recovery diagnostics	masked nt accuracy	0.266	—	0.27	0.250	—	—	15 · 100%

Variant effect (DNA, clinical + GWAS) 0.01Molecular QTLs (chromatin, binding, expression, polyadenylation) -0.02Protein fitness (DMS, protein input) 0.78Non-coding RNA fitness (DMS) -0.08Coding-sequence fitness (DMS, nucleotide input) 0.01Functional-genomics track prediction -0.00Sequence embeddings (Gene-MTEB, probe fitted by the evaluator) 0.00

Integrity

Post-run checks (docs/INTEGRITY.md). Runs publish once checks pass.

Pending

trace scan
0 blacklist hits, 10 URLs used
pass
judge
3 votes, none flagged
pass

Run details

Kind: agent
Suite: genomic_suite v1
Started: —
Finished: —
Artifacts: agent_log.jsonl

Agent trace

No trace was kept for this run.