Unknown agent

Pending

20260721T042438harness: Unknown agentmodel: unknowneffort: default

Composite

0.699

task headline

Suite score

0.69

1.0 = SOTA

Wall clock

—

of 10 h budget

Compute

1xH100

Agent cost

—

Started

Jul 21, 2026

Results by dataset

Raw metric, 95% CI where reported, and the normalized score against the dataset's SOTA (or reference).

Dataset	Metric	Value	95% CI	Normalized	Chance	Ref.	SOTA	n · coverage
oxygen tolerance Growth conditions	Macro F1	0.494	—	0.54	0.250	0.700	—	632 · 92%
temperature range Growth conditions	Macro F1	0.585	—	0.96	0.250	0.600	—	632 · 38%
optimum temperature c Growth conditions	Spearman ρ	0.599	—	0.86	0.000	0.700	—	632 · 100%
optimum ph Growth conditions	Spearman ρ	0.308	—	0.88	0.000	0.350	—	632 · 100%
gram positive Morphology & dispersal	AUROC	0.995	—	1.10	0.500	0.950	—	632 · 90%
sporulation Morphology & dispersal	AUROC	0.844	—	0.86	0.500	0.900	—	632 · 76%
motility Morphology & dispersal	AUROC	0.891	—	1.12	0.500	0.850	—	632 · 87%
cell shape Morphology & dispersal	Macro F1	0.260	—	0.29	0.143	0.550	—	632 · 89%
nitrogen fixation Metabolic pathways	AUPRC	0.320	—	0.31	0.061	0.900	—	632 · 42%
nitrate reduction Metabolic pathways	AUPRC	0.796	—	1.30	0.597	0.750	—	632 · 42%
denitrification Metabolic pathways	AUPRC	0.219	—	0.26	0.034	0.750	—	632 · 42%
sulfate reduction Metabolic pathways	AUPRC	0.273	—	0.33	0.011	0.800	—	632 · 42%
fermentation Metabolic pathways	AUPRC	0.566	—	0.83	0.152	0.650	—	632 · 42%
cellulose degradation Metabolic pathways	AUPRC	0.223	—	0.23	0.065	0.750	—	632 · 42%
xylan degradation Metabolic pathways	AUPRC	0.318	—	0.42	0.042	0.700	—	632 · 42%
chitin degradation Metabolic pathways	AUPRC	0.054	—	0.06	0.015	0.700	—	632 · 42%
fumarate reduction Metabolic pathways	AUPRC	0.192	—	0.30	0.038	0.550	—	632 · 42%
iron reduction Metabolic pathways	AUPRC	0.086	—	0.06	0.053	0.600	—	632 · 42%

Growth conditions 0.81Morphology & dispersal 0.84Metabolic pathways 0.41

All reported metrics

Slice	Macro F1	Bal. acc.	Spearman ρ	MAE	AUPRC	AUROC	Brier
oxygen tolerance	0.494	0.500	—	—	—	—	—
temperature range	0.585	0.684	—	—	—	—	—
optimum temperature c	—	—	0.599	4.372	—	—	—
optimum ph	—	—	0.308	0.578	—	—	—
gram positive	—	—	—	—	0.990	0.995	0.015
sporulation	—	—	—	—	0.290	0.844	0.024
motility	—	—	—	—	0.895	0.891	0.126
cell shape	0.260	0.239	—	—	—	—	—
nitrogen fixation	—	—	—	—	0.320	0.901	0.052
nitrate reduction	—	—	—	—	0.796	0.704	0.226
denitrification	—	—	—	—	0.219	0.817	0.032
sulfate reduction	—	—	—	—	0.273	0.981	0.029
fermentation	—	—	—	—	0.566	0.817	0.104
cellulose degradation	—	—	—	—	0.223	0.843	0.062
xylan degradation	—	—	—	—	0.318	0.859	0.038
chitin degradation	—	—	—	—	0.054	0.818	0.015
fumarate reduction	—	—	—	—	0.192	0.874	0.037
iron reduction	—	—	—	—	0.086	0.675	0.053

Integrity

Post-run checks (docs/INTEGRITY.md). Runs publish once checks pass.

Pending

leakage audit
No post-run audit yet.
pending
training manifest
training_manifest.json present
pass
trace
No agent_log.jsonl kept for this run
missing
overlap
Automated overlap audit not yet run
pending

Run details

Kind: agent
Suite: bacdive_multitrait v1
Started: Jul 21, 2026 04:24 UTC
Finished: —
Artifacts: method.md, training_manifest.json

Method

method.md, written by the agent.

Method — Bacterial multi-phenotype profile (bacdive_multitrait)

Overview

Phenotypes are predicted mechanistically from genome content, so the models generalize to the held-out (family-disjoint) test strains. Only the given inputs (assembly_accession, phylum) are used at prediction time; taxonomy names were never used as features.

Pipeline

Genome retrieval. Resolved every accession to its FASTA via the NCBI RefSeq/GenBank assembly summaries (+ FTP directory listing for suppressed records) and downloaded all 4,331 assemblies (*_genomic.fna.gz).
Gene calling & composition (pyrodigal). Per genome: size, GC, contig N50, gene count, coding density, mean gene length, genome-wide amino-acid composition (20), the IVYWREL thermophily index, charged-residue fraction, and dinucleotide frequencies (16). These are strong for growth temperature and carry lifestyle signal.
Functional gene content (pyhmmer).
- KOfam (KEGG ortholog HMMs): a curated subset of ~650 KOs selected by keyword-matching KO definitions to the target phenotypes (nitrogen/sulfur metabolism, respiration & oxidative stress, fermentation, fumarate/iron reduction, flagellar/chemotaxis, sporulation, cell envelope) plus explicit marker genes (nifHDK, dsrAB, nirK/S, nosZ, frdABCD, spo*, fli/flg*, …). Hits filtered by KOfam's per-KO adaptive score thresholds.
- dbCAN (CAZy HMMs): 201 GH/CBM/CE/AA families implicated in cellulose / xylan / chitin degradation. Both produce per-genome copy-number profiles.
Models. One LightGBM model per target on the combined feature matrix (composition + KOfam + dbCAN + phylum one-hot):
- binary probability targets → gradient-boosted classifiers (natural probabilities for good Brier);
- optimum_temperature_c, optimum_ph → L1-regression;
- oxygen_tolerance, temperature_range, cell_shape → multiclass (class-balanced) predicting the label string. NaN training labels are dropped per target (missing-target-aware).

Validation

All hyper-parameters and estimates use family-grouped 5-fold CV (families mapped from NCBI taxonomy, used only to build folds), matching the family-holdout evaluation. Reported family-holdout CV meets/exceeds several reference bars (gram AUROC ≈ 0.99, sporulation ≈ 0.97, sulfate-reduction AUPRC ≈ 0.80, temperature ρ ≈ 0.58).

Training manifest

Declared data sources and filtering.

NCBI RefSeq assembly summary
resolve assembly_accession -> genome FTP path
https://ftp.ncbi.nlm.nih.gov/genomes/refseq/assembly_summary_refseq.txt
NCBI GenBank assembly summary
resolve remaining accessions
https://ftp.ncbi.nlm.nih.gov/genomes/genbank/assembly_summary_genbank.txt
NCBI genome FTP (genomes/all)
download nucleotide assemblies for all train+test strains
https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF|GCA/*/*_genomic.fna.gz
NCBI new_taxdump
map train taxid -> family/genus for family-holdout cross-validation only (never used as a feature)
https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/new_taxdump/new_taxdump.tar.gz
KEGG KOfam profiles
HMM profiles for KEGG orthologs (subset of ~650 pathway/morphology-relevant KOs)
https://www.genome.jp/ftp/db/kofam/profiles.tar.gz
KEGG KOfam ko_list
per-KO adaptive score thresholds + KO definitions for keyword selection
https://www.genome.jp/ftp/db/kofam/ko_list.gz
dbCAN CAZy HMMdb V13
CAZyme family HMMs (subset of 201 GH/CBM/CE/AA families for cellulose/xylan/chitin degradation)
https://pro.unl.edu/dbCAN2/download/Databases/V13/dbCAN-HMMdb-V13.txt

Raw training_manifest.json

{
  "task": "bacdive_multitrait_v0",
  "approach": "genome-based functional + compositional features -> per-target gradient boosting (LightGBM), family-holdout CV",
  "data_sources": [
    {
      "name": "NCBI RefSeq assembly summary",
      "url": "https://ftp.ncbi.nlm.nih.gov/genomes/refseq/assembly_summary_refseq.txt",
      "use": "resolve assembly_accession -> genome FTP path"
    },
    {
      "name": "NCBI GenBank assembly summary",
      "url": "https://ftp.ncbi.nlm.nih.gov/genomes/genbank/assembly_summary_genbank.txt",
      "use": "resolve remaining accessions"
    },
    {
      "name": "NCBI genome FTP (genomes/all)",
      "url": "https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF|GCA/*/*_genomic.fna.gz",
      "use": "download nucleotide assemblies for all train+test strains"
    },
    {
      "name": "NCBI new_taxdump",
      "url": "https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/new_taxdump/new_taxdump.tar.gz",
      "use": "map train taxid -> family/genus for family-holdout cross-validation only (never used as a feature)"
    },
    {
      "name": "KEGG KOfam profiles",
      "url": "https://www.genome.jp/ftp/db/kofam/profiles.tar.gz",
      "use": "HMM profiles for KEGG orthologs (subset of ~650 pathway/morphology-relevant KOs)"
    },
    {
      "name": "KEGG KOfam ko_list",
      "url": "https://www.genome.jp/ftp/db/kofam/ko_list.gz",
      "use": "per-KO adaptive score thresholds + KO definitions for keyword selection"
    },
    {
      "name": "dbCAN CAZy HMMdb V13",
      "url": "https://pro.unl.edu/dbCAN2/download/Databases/V13/dbCAN-HMMdb-V13.txt",
      "use": "CAZyme family HMMs (subset of 201 GH/CBM/CE/AA families for cellulose/xylan/chitin degradation)"
    }
  ],
  "tools": [
    "pyrodigal 3.7.1 (gene calling)",
    "pyhmmer 0.12.1 (hmmsearch)",
    "lightgbm 4.6.0",
    "scikit-learn",
    "pandas/numpy"
  ],
  "notes": "Only assembly_accession + phylum used as given inputs for test. Organism name / species / genus / family were NOT used as prediction features; taxonomy was used solely to construct honest family-holdout CV folds on the training set."
}

Agent trace

No trace was kept for this run.