On Twitter last month, two of my colleagues in the American Theatre Critics Association tweeted an article from the New York Daily News stating that singing musical theatre songs helped “stave off the effects of” Alzheimer’s disease.

Sounds great, right? As an arts lover and a psychologist, I was intrigued, so I decided to click the link. It turns out that the full story is much more complicated than what the tweets or the New York Daily News state.

The New York Daily News article was published in November 2013 and is based on a Guardian article published the previous day. The Guardian story is more detailed but tells essentially the same story: elderly adults who participated in music classes in their assisted living facilities scored higher on cognitive tests than they did before the music lessons. The lessons lasted 50 minutes each and occurred 3 times per week for months. And the results are promising, according to the Guardian:

The sessions appeared to have the most striking effect on people with moderate to severe dementia, with patients scoring higher on cognitive and drawing tests, and also on a satisfaction-with-life questionnaire at the end of the study.

The Guardian article is based on a poster that the study’s first author, Linda E. Maguire of George Mason University, presented at a conference of the Society for Neuroscience. For almost 18 months the popular press and a one-paragraph summary were the entire public record of the study. The study was published in April 2015 in the Journal of the American Geriatrics Society. (A delay of this length is not unusual in scholarly publishing.) The full scholarly article provides very little evidence that, “Singing show tunes helps fight off dementia” (in the words of the New York Daily News).

Reality Check

There are several reasons I am skeptical of this study. First, the sample size is 45 older adults (85% female, mostly Caucasian) who all lived in the same assisted living facility and were at least 70 years old. There is little reason to expect these results to apply to elderly adults who live independently or people who are younger than the study participants. Additionally, only 18 people in the study had dementia. With such small group sizes, the study has low statistical power, which is the ability for a study to detect a real effect of an intervention. This means that any effect that is detected is more likely to just be a fluke.

Another major problem of this study is that there is no equivalent control group. The researchers compared people with dementia and people without dementia. Without a control group of people with dementia that do not experience the singing lessons, there is no way to know what would have happened to the residents with dementia if they had not had the singing lessons. Therefore, it is not possible to say whether the improvements in mental functioning are the result of the singing lesson or other events happening at the nursing home.

There also is not a lot of singing in the lessons. In one video of the lessons that the authors provide, the actual singing is about one-fourth of the video. If this video is typical of the singing intervention, then each lesson is a consists of about 12½ minutes of singing. Yet, the authors want us to believe that this short amount of singing can provide major improvements in mental functioning. (Additionally, they don’t report enough detail to state exactly the size of the improvements, which is another flaw of the study.) But the general rule of thumb in psychology is that major changes in behavior require major interventions, and that short/weak interventions produce little or no change in human behavior. It’s not clear why singing should buck this general trend in psychology.

Additionally, in the full study, the authors do not make any claim that there is anything special about musical theatre music. In fact, musical theatre was just one of four genres that the music focused on: Valentine’s, patriotic, musical theatre, and folk/patriotic music. There also seems to have been some religious songs included in the music program.

Another odd characteristic of the study is that while people with dementia had higher functioning at the end of the four months, nursing home residents without dementia got worse after taking the singing sessions. If anything, this study demonstrates that most elderly people should avoid singing because it may make their mental functioning worse!

Finally, the goal of the singing lessons were “. . . to meet cognitive, emotional, and linguistic goals.” Yet, the authors never reported any emotional or language outcomes for the patients. In psychological research, is it deficient to collect data on outcomes without reporting the results. Arguing that music lessons are beneficial without reporting all the data makes it difficult for readers to evaluate whether the lessons are truly beneficial or not.

Lessons from the Study

Is Maria preventing dementia in the von Trapp children? Probably not. According to Maguire and her coauthors’ study, music lessons provide no benefit for people without dementia.

This study is not worthless. It is likely true that mental stimulation is beneficial for people with dementia. Most psychologists who specialize in aging recommend activities that require elderly people to be mentally engaged (e.g., crossword puzzles, playing bridge, reading). Taking singing lessons could plausibly be a beneficial activity for elderly people. But the popular media’s description of the study is oversimplified.

Beyond the findings here, the study has important lessons for readers. First, be cautious of journalistic accounts of scientific studies. It is a journalist’s job to take a complex story and retell it in an understandable way. Most are extremely good at this, but sometimes the result is too simplified, as it was in the Guardian and New York Daily News articles.

Second, go to the original source for research. Journalistic media are a great way of learning of the existence of research, but nothing beats reading the actual article to understand the research. The best research is published in reputable peer-reviewed scholarly journals. Peer review doesn’t guarantee that the research is good, but it does serve as a quality control mechanism for research. Sometimes it is hard to access these articles, but if you contact the corresponding author (whose email is always listed on the web page for the article), most are willing to email you a copy.

Finally, if the results seem too good to be true, they probably are. If the headline sounds like clickbait, it probably is. If you want the results to be true or if the study meshes well with your social or political beliefs, then you should be extra skeptical. Scientific research is usually messy and complicated, and it rarely tells a completely clean, simple story.

References

Maguire, L. E., Wanschura, P. B., Battaglia, M. M., Howell, S. N., & Flinn, J. M. (2015). Participation in active singing leads to cognitive improvements in individuals with dementia. Journal of the American Geriatrics Society, 63, 815-816. doi:10.1111/jgs.13366