A study of the BJ’s Wholesale Club employee wellness program attracted a lot of attention in the media, but the most important facts about the study were overlooked.
“The model aims to answer the question: what is the effect of offering an individual the opportunity to participate in a wellness program?”
— From the study’s supplement (eMethods 3. Statistical Analysis)
Facty Fact 1: Worksites, Not Workers, Were Randomized
The BJ’s study was not primarily an evaluation of participation outcomes:
“In our primary analyses, we estimated the effect of working at a treatment worksite on outcomes, regardless of participation in the wellness program.”
This was a study of how exposure to (not participation in) a wellness program affected wellbeing. 20 BJ’s stores were randomly selected to offer the wellness program to their employees. 140 stores did not offer the program and were used as controls. The researchers compared outcomes for all employees at stores that offered the program to outcomes for all employees at the control stores that didn’t offer the program.
At first blush, it may seem bizarre to study “exposure” to a wellness program rather than “participation in” a wellness program. But it’s a compelling design. After all, employers are interested in the overall impact a program has on their employees and/or their business, not just on the people in the program (especially if participation rates are low). What’s more, the design assured that self-selection bias (e.g. more motivated workers being drawn to participate) didn’t skew results.
Just for Good Measure…
The investigators, Zirui Song and Katherine Baicker, assessed program participation outcomes, too, using complex statistical methods to filter out self-selection bias. (Like last year’s Illinois University study, Song and Baicker also did a participant/non-participant analysis — commonplace in wellness studies — demonstrating again that it produces deceptively positive results.)
Facty Fact 2: Culture Wasn’t Completely Overlooked
Did Song and Baicker have culture in mind when they designed their study? They wrote:
“Unlike the Illinois study, this intervention was implemented at the worksite level (rather than varying across individuals within the same worksite), perhaps better facilitating changes in workplace culture and providing greater social supports for behavior change.”
Regrettably, the intervention included no discernible effort to influence the workplace culture. It seems there was an assumption that the very existence of a wellness program might modify culture. But, again, in the real world, this is an assumption employers often make. So it’s fair game.
Facty Fact 3: The Study Looked Beyond Healthcare Costs
Health behavior and healthcare cost outcomes are relevant because they remain the primary motivations for employers’ wellness programs. Regardless, Song and Baicker, did dig deeper. Study metrics included health-related quality of life based on SF-8 scores, depression scores from the PHQ-2, job performance based on performance evaluations, and absenteeism (along with the usual healthcare and self-reported health behavior data). These metrics go beyond healthcare costs and may give insight into employees’ overall wellbeing.
Facty Fact 4: The Program Was Better Than Apologists Would Have You Believe
The study authors did themselves a disservice by referring to the program as consisting of 8 “modules.” Readers assume these were educational sessions. They were, in fact, multi-week interventions — some including fashionable social features and gamification — that touched on a host of topics including healthcare consumerism/self-care, fitness, nutrition, stress, sleep, meditation, weight management (supported with weekly coaching by a dietitian), unplugging from electronic media, coloring, and yoga — with much of the activity apparently taking place independently on employees’ own time — in addition to biometric screenings and health coaching.
The modules were customized and facilitated by a registered dietitian, who also led employees in group fitness activities and cooking demonstrations.
Let’s not confuse the wellness ideals we espouse at conferences with workplace reality. The BJ’s program is typical of what many employers present to their employees as wellness.
While I’ll never draw a petty conclusion like “wellness works” or “wellness doesn’t work,” I do think the design of this study should prompt important learnings, discourse, and, yes, further research.
Addendum: Burying the Lead
There’s something potentially more important, and more overlooked, about the BJ’s study:
“About 20% of the population was black and 18% Hispanic. Full-time workers comprised about 60% of the study population. Mean earnings for full-time salaried workers was slightly under $50,000 per year, and full-time hourly workers earned about half that amount.”
These demographics are likely typical of retail workers, the largest employment sector in the US (followed by food-service). Ultimately, we in the wellness industry must ask ourselves whether our programs are really being developed to serve these workers. And if they’re not — regardless of how holistic, robust, and sustainable they may be — how will wellness programs ever achieve positive outcomes?
Effect of a Workplace Wellness Program on Employee Health and Economic Outcomes: A Randomized Clinical Trial. Abstract. You may be able to sign up for access to the full article.
The Impact of Workplace Wellness on Health, Health Care, and Employment Outcomes: A Randomized Controlled Trial (The analysis plan, which includes detailed descriptions of the intervention in Appendix I)
[Edited on June 20, 2021 to update the link to SF-8 information.]