The goal is to develop clinically actionable biomarkers, that should generalize to ‘unseen’ data, so we have marked 30% of clinical and treatment outcome data as REPLICATION. These are blinded for prospective validation and replication purposes. Interested researchers can share () their diagnostic or treatment-outcome predictions on these REPLICATION data with us, so we can perform a ‘blinded out-of-sample’ validation for them, and post the results on this page, so that reviewers for journals can also consult this independent validation of their algorithms. Conversely, reviewers or editors faced with ‘too good to be true’ claims, can refer researchers to this resource, to perform validations and share them with us for verification and publication. We hope this will accelerate the discovery and validation of clinically actionable biomarkers that we hope will soon benefit patients and improve psychiatric and neurological care in the clinic.
We’re using nPPV as primary outcome, the table below is sorted with best results on top. Our focus is on ADHD and MDD, so OCD results are omitted.
| First author | Year | Original Reference | EEG Method Used | Diagnostic/Prognostic | Replicated? | PPV | nPPV | NPV | nNPV | R2 | t | F | p | ES (Cohen's d; -=opposite direction) | Balanced accuracy |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Sama Therapeutics, Ahmed et al | 2023 | ADHD Neurofeedback remission | prognostic | yes | 83% | 127% | 38% | 58,6% | 57% | ||||||
| Sama Therapeutics, Ahmed et al | 2023 | MDD TMS remission | prognostic | yes | 70% | 123,1% | 52% | 90,8% | 61% | ||||||
| Sama Therapeutics, Ahmed et al | 2023 | ADHD Neurofeedback response | prognostic | yes | 100% | 121,4% | 38% | 45,5% | 82% | ||||||
| Sama Therapeutics, Ahmed et al | 2023 | MDD TMS response | prognostic | yes | 77% | 111,1% | 54% | 77,8% | 64% | ||||||
| Alekseichuk, University of Minnesota | 2023 | Deep Neural Network | prognostic | no (X) | 72% | 100.7% | 31% | 43.2% | |||||||
| Alekseichuk, University of Minnesota | 2023 | Prediction rTMS over prefrontal cortex | prognostic | no (X) | 71% | 99.3% | 28% | 22.2% | |||||||
| Nouminous (Dilip Rajeswari) | 2022 | TMS remission | prognostic | no (X) | 50% | 91.7% | 41% | ||||||||
| Roelofs et al. | 2020 | Corlier et al. 2019 Brain Stimul. | Individal alpha peak frequency | prognostic | yes (V) | 0.162/0.026 | 0.022 | ||||||||
| Bailey et al. | 2020 | Bailey et al. 2019 J Affect Disord | Alpha power | prognostic | no (X) | 0.851 | 0.357 | ||||||||
| Bailey et al. | 2020 | Bailey et al. 2019 J Affect Disord | EEG theta connectivity | prognostic | no (X) | 1.638 | 0.103 | 0.2524 | |||||||
| Krepel et al. | 2019 | Krepel et al. 2019 Psych Med. | Neo-NFFI Openness | prognostic | no (X) | 1.49 | 0.227 | -0.33 | |||||||
| Krepel et al. | 2019 | Krepel et al. 2019 Psych Med. | BDI Anhedonia scale | prognostic | yes (V) | 60.7% | 80% | 8.516 | 0.005 | 0.76 | |||||
| Krepel et al. | 2018 | Arns et al. 2012 Brain Stimul. | Larger P300 | prognostic | no (X) | 0.008 | 0.929 | 0.019 | |||||||
| Krepel et al. | 2018 | Arns et al. 2012 Brain Stimul. | Low individual alpha peak frequency | prognostic | no (X) | 0.448 | 0.054 | 0.19 | |||||||
| Krepel et al. | 2018 | Arns et al. 2012 Brain Stimul. | High fronto-central theta power | prognostic | no (X) | 0.753 | 0.388 | 0.173 | |||||||
| First author | Year | Original Reference | EEG Method Used | Diagnostic/Prognostic | Replicated? | Sensitivity | Specificity | R2 | t | F | p | ES (Cohen's d; -=opposite direction) | Balanced accuracy | ||
| Van Strien | 2022 | Prediction EEGNet | diagnostic | yes (V) | 75% | 140.5% | 69% | 128.5% | |||||||
| Van Strien | 2022 | Prediction DeprNet | diagnostic | yes (V) | 67% | 126% | 61% | 113.8% | |||||||
| Van Strien | 2022 | Prediction Logistic Regression | diagnostic | no (X) | 49% | 91.1% | 44% | 82.3% | |||||||
Best result!