One pip install, one neuralbench command, one submission file. The start-kit ships the same evaluation harness the organizers use, every baseline reported on the leaderboard, and a deterministic toy split you can iterate on before the warm-up opens.
1# 1. Install the start-kit2$ pip install neuralbench[full]3# 2. Run the toy split end-to-end5$ neuralbench demo --track bci --model eegnet6↳ trained · scored · submission.parquet written
The path is identical for every track. Change the track flag, swap the model, and the rest of the pipeline stays the same.
pip install neuralbench[full] pulls in eegdash, braindecode, MNE, and the evaluator. Use [lite] if you only want the data loaders without the model zoo.
EEGDash streams the BIDS layout — only the recordings you reference are downloaded. The warm-up split for every track lives under ~/.cache/eegdash/ after the first call.
Every track ships at least one baseline you can train with a single neuralbench train call. Configs are plain YAML — copy one, edit the model, keep the same data + scoring blocks.
The evaluator is the exact code we run on Codabench. Run it locally first — neuralbench score writes a submission.parquet with predictions, hyperparameters, and the git SHA of your model code for the audit phase.
Drop the submission.parquet on the Codabench page for your track. Five submissions per team per day during the sealed phase. Final ranking is the best of your last five submissions.
The same pattern works for every track: load with EEGDash, train with Braindecode, score with NeuralBench. Swap the dataset id and the model — the rest of the pipeline stays the same.
1from eegdash import EEGDashDataset2from braindecode.models import EEGNetv43from neuralbench import Trainer, Evaluator4ds = EEGDashDataset(track="bci", split="warmup")6train, val = ds.split(by="subject", ratio="4:1")7model = EEGNetv4(9n_chans=ds.n_chans,10n_classes=ds.n_classes,11input_window=2.0,12)13trainer = Trainer(model, epochs=100, lr=1e-3)15trainer.fit(train, val)16# Identical scoring to Codabench18sub = Evaluator("bci").predict(model, ds.test)19sub.to_parquet("submission.parquet")
1# Pull the BCI warm-up split2$ neuralbench data bci --split warmup3# Train the EEGNet baseline5$ neuralbench train bci -m eegnet --epochs 1006# Score and write submission file8$ neuralbench score bci --out submission.parquet9↳ bal. acc 0.586 · 20 subjects · 120 sessions10# Reproduce a previous run from its config12$ neuralbench replay configs/bci/eegnet.yaml
Numbers below are the warm-up split — the public split you'll use for development. The sealed test set is held out by the organizers and only opens Aug 1, 2026.
You don't upload code during the public phase — you upload predictions. Top entries are pulled for a reproducibility audit at test-freeze: we re-run your neuralbench replay against the sealed split. The submission file's schema is fixed and validated client-side before upload.
neuralbench score
neuralbench check submission.parquet
Schema spec on GitHub →
There is no compute cap during training. The sealed-test inference budget is the only constraint — your scoring container must finish a full test pass in under 60 minutes on a single A100. AWS gives finalists complimentary instance hours; details land in the awards page.
Public, redistributable datasets are fair game. Closed datasets and the sealed test split are not. Declare every external corpus you used in the 2-page methods note that ships with your final submission — the audit cross-checks this against your neuralbench replay.
Top-3 per track go through a reproducibility audit on Oct 1, 2026. We re-run your training pipeline from the committed config, then score the resulting weights against the sealed test split. If the score is within tolerance of your submission, you stay on the board. If not, you drop.
Track 5 requires a single shared EEG encoder reused across the EEG-to-IMG, BCI, and Sleep tracks. Heads can be track-specific. Encoders fine-tuned on the sealed test set or trained per-track separately are rejected at audit.
Clone the start-kit, run the toy split, and you're on the leaderboard in fifteen minutes. Everything is MIT-licensed — fork freely, but please cite the white paper and the 2025 challenge that this work continues.
1$ git clone https://github.com/neural-interfaces26/neuralbench2$ cd neuralbench3$ pip install -e ".[full]"4# 30-second sanity check6$ neuralbench demo --track bci --toy