TDBRAIN Challenge

The goal is to develop clinically actionable biomarkers, that should generalize to ‘unseen’ data, so we have marked 30% of clinical and treatment outcome data as REPLICATION. These are blinded for prospective validation and replication purposes. Interested researchers can share () their diagnostic or treatment-outcome predictions on these REPLICATION data with us, so we can perform a ‘blinded out-of-sample’ validation for them, and post the results on this page, so that reviewers for journals can also consult this independent validation of their algorithms. Conversely, reviewers or editors faced with ‘too good to be true’ claims, can refer researchers to this resource, to perform validations and share them with us for verification and publication. We hope this will accelerate the discovery and validation of clinically actionable biomarkers that we hope will soon benefit patients and improve psychiatric and neurological care in the clinic.

We’re using nPPV as primary outcome, the table below is sorted with best results on top. Our focus is on ADHD and MDD, so OCD results are omitted.

First authorYearOriginal ReferenceEEG Method UsedDiagnostic/PrognosticReplicated?PPVnPPVNPVnNPVR2tFpES (Cohen's d; -=opposite direction)Balanced accuracy
Sama Therapeutics2023ADHD Neurofeedback remissionprognosticyes83%127%38%58,6%57%
Sama Therapeutics2023MDD TMS remissionprognosticyes70%123,1%52%90,8%61%
Sama Therapeutics2023ADHD Neurofeedback responseprognosticyes100%121,4%38%45,5%82%
Sama Therapeutics2023MDD TMS responseprognosticyes77%111,1%54%77,8%64%
Alekseichuk, University of Minnesota2023Deep Neural Networkprognosticno72%100.7%31%43.2%
Alekseichuk, University of Minnesota2023Prediction rTMS over prefrontal cortexprognosticno71%99.3%28%22.2%
Numinous2022TMS remissionprognosticno50%91.7%41%
Roelofs et al.2020Corlier et al. 2019 Brain Stimul.Individal alpha peak frequencyprognostic yes0.162/0.0260.022
Bailey et al.2020Bailey et al. 2019 J Affect DisordAlpha powerprognosticno0.8510.357
Bailey et al.2020Bailey et al. 2019 J Affect DisordEEG theta connectivityprognosticno1.6380.1030.2524
Krepel et al.2019Krepel et al. 2019 Psych Med.BDI Anhedonia scaleprognosticyes60.7%80%8.5160.0050.76
Krepel et al.2018Arns et al. 2012 Brain Stimul.Larger P300prognosticno0.0080.9290.019
Krepel et al.2018Arns et al. 2012 Brain Stimul.Low individual alpha peak frequencyprognosticno0.4480.0540.19
Krepel et al.2018Arns et al. 2012 Brain Stimul.High fronto-central theta powerprognosticno0.7530.3880.173
First authorYearOriginal ReferenceEEG Method UsedDiagnostic/PrognosticReplicated?SensitivitySpecificityR2tFpES (Cohen's d; -=opposite direction)Balanced accuracy
Van Strien2022Prediction EEGNetdiagnosticyes75%140.5%69%128.5%
Zwaans2024CNN LSTMdiagnosticyes 77%100.5%24%98.3%50%
Sama Therapeutics2023diagnosticno57%37%47%

Best result!