The Fallacy of Sham-Controlled Neurofeedback Trials: A Reply to Thibault and Colleagues (2018). - PubMed - NCBI
[...]
Learning Methodology Matters
Table 1 summarizes the methodology and findings from six sham-controlled trials treating ADHD. Although each study acknowledged NFB is based on operant learning,
their methodology violated established learning science by using either automated or manually adjusted EEG reward thresholds to maintain an “about 80%” level of reward across sessions and subjects. This procedure is contrary to basic learning principles. First, operant conditioning targets a response followed by a stimulus-event to make the desired response occur more or less frequently and then plots the target response’s occurrence over time to document whether or not learning has occurred.
In these studies, the target response was not consistently calculated, monitored, plotted, and presented to NFB subjects. Therefore, it is not known what response (if any) was conditioned. Second, the studies do not reference the effects of practice in the experimental process. Subjects in both groups engaged in the same set of behaviors during sessions (e.g., maintaining stillness and focus, reducing muscle and eye-movement artifacts, relaxation, posture, and breathing). If subjects did not engage in these practiced behaviors, their EEG data were riddled with artifact and worthless. Third, operant conditioning of the EEG requires that these core concepts are strictly adhered to demonstrating the operant behavior has been learned and such documentation of learning should occur before examining outcome measures of interest (Cannon, 2015).
In these studies, every reset of the EEG reward threshold delivered operant consequences to subjects’ brains antithetical to the goal of training. As Pigott and colleagues (2017) note, if the targeted EEG was strengthening, reinforcement was withdrawn and reset down to 80% thereby punishing participants for learning to self-modulate. Conversely, if the targeted EEG was decreasing, participants were reinforced up to 80% thereby rewarding them for decreasing its strength. (p. 897)
At every reset of the reward threshold, NFB subjects therefore were either rewarded for not learning to self-modulate the targeted EEG or administered a Type 2 punishment for the beginnings of success.
Given their flawed methodology, it is not surprising that all six studies found:
- No evidence NFB subjects learned to self-modulate the targeted EEG;
- No separation between NFB and sham feedback on any outcome measure; and
- When assessed, the vast majority (71% to 75%) of NFB subjects thought they received sham-feedback—correctly determining the NFB they received was often false.
Intriguingly, four of the studies also found significant improvement in both groups, leading Thibault and colleagues among many others to argue that these beneficial effects are due to placebo phenomena versus any specific effects from NFB. Two points in response below:
First, flawed methodology prevented NFB subjects from learning to self-modulate the targeted EEG and therefore no specific effects should be expected since each study compared two forms of false-feedback. Second, both groups participated in an active intervention. Ninaus and colleagues (2013) found multiple cortical regions of the brain are activated when blinded subjects were told to focus and try to control randomly moving bars during five 20-s rounds. In contrast, no such changes occurred when subjects were instructed to merely watch the moving bars. Subjects in sham-controlled trials are commonly instructed to sit still, focus, and use their brains to increase positive feedback. Similar cortical regions therefore likely underwent a vigorous workout during subjects’ 30+ sessions sitting still and trying to control that which was uncontrollable.
This is hardly a “placebo” intervention as traditionally understood and likely only had positive effects because subjects were deceived into believing they had a 50% chance of receiving accurate EEG feedback. Transparency eliminated the brain activation found by Ninaus et al. as it likely would in all false-feedback trials.
Thibault and colleagues’ (2018) claim that NFB is a placebo is not supported by the referenced data. Their referenced studies compared two forms of false-feedback—
not operant conditioning of the EEG. NFB has a 75+ year history of scientific inquiry documenting operant conditioning of the EEG in cats (e.g., Wyrwicka & Sterman, 1968), primates (e.g., Schafer & Moore, 2011), and people (e.g., Jasper & Shagass, 1941), including a 40-year history of research treating ADHD children (Lubar & Shouse, 1976; Shouse & Lubar, 1979). The authors though dismiss this extensive research history asserting that “Following the results from recent double-blind studies, we can now add EEG-nf for ADHD to this list of placebo therapies that masquerade under other biomedical labels” (p. 2). In contrast, it is our assessment that it is these double-blind studies themselves that are the masquerade since
they did not compare operant conditioning of the EEG with a sham-control but rather two forms of false-feedback.
[...]
Finally, the NFB field and its detractors continue to conduct research that violates behavioral principles, and both sides cite such substandard research when it supports their 8
Journal of Attention Disorders 00(0) viewpoint. This practice must stop.
Evidence of learning trumps all, and if there is no evidence of learning, operant conditioning of the EEG did not occur.
