Today I’ll be reviewing Lumosity’s comprehensive scientific study into the effectiveness of its brain training programs for the general population. The study was published in Sept 2015 in PLoS One (full paper here).

The Lumosity study

The study can’t be faulted for the number of participants in the study – what is referred to technically as its statistical power. There were 4,715 participants, covering all age groups.

Participants were divided into a brain training group and a control group. The brain training group completed a 15-minute Lumosity session at least 5 days per week for 10 weeks (a total of 12.5 hours of training in total) while the control group completed crossword puzzles.

The following cognitive abilities were assessed before and after training on psychometric tests:

• Forward Span and Reverse Memory Span for short term / working memory.
• Grammatical Reasoning and Arithmetic Reasoning  – two different measures of logical reasoning.
• Progressive Matrices – a measure of IQ (fluid intelligence).
• Go/No-Go for decision-making speed.
• Two-Target Search – a measure of visual attention.

The results of the Lumosity study

Overall, the Lumosity training compared to doing crossword puzzles improved performance in all cognitive abilities except for the visual attention measure (no difference) and the grammatical reasoning measure where doing cross-words actually resulted in a bigger improvement than Lumosity brain training.. This is shown in the graph below.

How meaningful are the brain training  improvements?

The graph shows statistically significant improvements in most of the test scores, including IQ, but what does this mean for practical purposes? How meaningful are the improvements for helping us perform better cognitively? To answer this we need some measure of what is called the ‘effect size’ – i.e. how big are the improvements on a scale that makes sense to us?

Something that makes sense to many of us are points on an IQ scale. The study reports effect sizes on exactly this kind of scale for all the cognitive abilities. These are the point gains from Lumosity brain training compared to doing crosswords.

• Forward Span………………….2.3 points
• Working memory…………….1.7 points
• Grammatical Reasoning….. -1.2 points
• Arithmetic Reasoning……….3.7 points
• Progressive Matrices (IQ)….1.7 points
• Speeded Decision-making….2.4 points
• Visual attention………………..0.5 points (not significant)

IQ and working memory gains: under 2 points

The two critical measures for overall cognitive ability are IQ and working memory.

• Compared to doing cross-word puzzles, 10 weeks of  Lumosity training results in a 1.7 point IQ gain.

Put in these more meaningful terms, that’s not too impressive.

We get exactly the same small gain for working memory, a measure of our ‘mental workspace’ that is important for our attention control and ability to multi-task.

But here we are looking at the brain training gains compared to doing crossword puzzles. It could be that doing crosswords is good for IQ, so we need to look at the absolute IQ gain – not relative to some other kind of mental task. Looking at absolute gains is a better measure of what we can expect from Lumosity brain training.

• The absolute IQ gain from Lumosity training was 3 IQ points.

For working memory, the absolute gain was 2.6 points.

And Since we are talking statistical averages here and there is a wide spread of scores, there’s a large percentage of users who won’t see any gains at all, and will in fact see the reverse.

How does Lumosity training compare to IQ Mindware training?

IQ Mindware apps are based on optimized dual n-back training with interference control. N-back training is a type of working memory training – a well established brain training method in the scientific literature (reference).

• Basic (non-optimized) n-back training has been shown in meta-reviews of 30 independent studies to result in an absolute IQ gain from pre- to post-training of 6.2 IQ points.

(The reference for this ‘effect size’ result can be found in Table 1 here – the relevant statistic is TxES.)

Compared to Lumosity training for a similar duration, that’s more than twice the IQ gain. And there are even more impressive results for working memory gains, since n-back training is actually a type of working memory training.

And with HighIQPro Mindware, the dual n-back is optimized for IQ and working memory gains, incorporating interference control which is the primary ability linking working memory with IQ (reference).

Summary

1. Brain training works

The comprehensive study on Lumosity training, shows that the brain training does in fact result in brain performance gains. This puts rest to the idea that computer-based brain training is a waste of time which was the initial consensus among the scientific community (reference).

2. Brain training only works with a lot of time invested

12 and a half hours over 10 weeks of Lumosity training is a lot of training, and more than is typical of Lumosity users. This could explain why studies looking at more typical Lumosity training patterns have found no benefits whatsoever (e.g. Nature article).

IQ Mindware training also requires this kind of training duration.

There is no escaping it – for real brain benefits, much as there is no quick way to getting fit or strong at a gym.

3. Lumosity training results in small (2-3 point) training effects

Lumosity training results in IQ, working memory and other cognitive ability improvements on the order of 2-3 points – provided you train for 12 and a half hours over many weeks.

The small gains, it is reasonable to conclude, are partly the result of the type of ‘multiple game’ approach, where there is no specific focus on working memory training which is the most scientifically well-established brain training method (reference). It is for this reason that the more scientifically rigorous brain training providers (e.g. CogMed) focus on working memory training.

4. Working memory based training with IQ Mindware apps results in greater than twice the training effects of Lumosity

Training effects for standard, lab based working memory training such as the dual n-back result in a greater than 6 point IQ gain. When this training method is optimized for cognitive performance gains by improving player incentives (gamification), and adding interference control, the gains are found to be significantly higher than this. It is for this reason that IQ Mindware guarantees a 10-20 point IQ gain on standardized IQ tests.

Combining IQ Mindware training with other strategies such as exercise and intermittent fasting can further optimize brain performance gains.