Built on Careful Science
We describe how we test our models and publish the results as they are, including the ones that are not good yet.
Validation Approach
LOSO Cross-Validation
Leave-one-subject-out cross-validation goes through every subject in the dataset, training on all the others and testing on the one held out. No data from the test subject reaches training, which is the closest a retrospective study gets to meeting a new patient.
External Testing
Internal results are only half the picture, so we also test on cohorts the model has never seen. On the Healthy Brain Network cohort (N=321, ages 5 to 17), Neuraxis-Child scored AUC 0.519, which is chance level. We report that result here because it is the most important limit of the current model.
Why These Choices Matter
Much of the published EEG-ML literature reports inflated performance using k-fold cross-validation, which can leak within-subject information across folds. LOSO gives more realistic estimates. Testing on an independent cohort shows whether a model has learnt something about the condition or only about one dataset.
Performance Results
All results come from LOSO cross-validation. They are preliminary research metrics, not clinical performance claims.
Neuraxis-Child v1.0
Prototype · Research Use OnlyAUC
0.650
Balanced Accuracy
0.595
Validation
LOSO CV
Dataset
IEEE ADHD
Age Range
7-12 years
Neuraxis-Adult v1.0
Under revalidation · Research Use OnlyAUC
Not reported
Balanced Accuracy
Not reported
Validation
LOSO CV
Dataset
TDBRAIN (N=72)
Age Range
18-35 years
Internal LOSO results on the IEEE cohort. On an independent external cohort (HBN-EEG, N=321, ages 5 to 17) Neuraxis-Child scored AUC 0.519, which is chance level. All reported metrics are preliminary research results. Research Use Only (RUO).
Research Datasets
IEEE ADHD Dataset
Full dataset · Ages 7-12Public paediatric EEG dataset with clinical ADHD diagnoses. Chosen because its labels are clinician-confirmed diagnoses (not self-report) and it has been used in other published ADHD-EEG studies, so results can be compared.
TDBRAIN
N=72 subset · Ages 18-35Lifespan EEG database (ages 5 to 89) from the Brainclinics Foundation, Nijmegen, The Netherlands (van Dijk et al., 2022). Used to study ADHD in adults recorded on different equipment from the paediatric data. The 72-subject subset covers the adult ADHD age range; the adult module is under revalidation.
HBN-EEG
N=321 · Ages 5-17 · External test onlyHealthy Brain Network resting-state EEG, 105 ADHD and 216 control. Kept completely separate from training and used only to test whether Neuraxis-Child generalises to a new, community-recruited cohort.
Publications
We are preparing our first publications. Updates will be shared here and on our LinkedIn page.
Commitment to Open Science
All our validation uses publicly available datasets. We report conservative LOSO metrics rather than optimistic k-fold estimates, and we document our methods so others can check them.
Working in EEG research or clinical neuroscience?
We are looking for research and clinical partners to help validate SynapCore.