A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Katherine L Milkman; Linnea Gandhi; Mitesh S Patel; Heather N Graci; Dena M Gromet; Hung Ho; Joseph S Kay; Timothy W Lee; Jake Rothschild; Jonathan E Bogard; Ilana Brody; Christopher F Chabris; Edward Chang; Gretchen B Chapman; Jennifer E Dannals; Noah J Goldstein; Amir Goren; Hal Hershfield; Alex Hirsch; Jillian Hmurovic; Samantha Horn; Dean S Karlan; Ariella S Kristal; Cait Lamberton; Michelle N Meyer; Allison H Oakes; Maurice E Schweitzer; Maheen Shermohammed; Joachim Talloen; Caleb Warren; Ashley Whillans; Kuldeep N Yadav; Julian J Zlatev; Ron Berman; Chalanda N Evans; Rahul Ladhania; Jens Ludwig; Nina Mazar; Sendhil Mullainathan; Christopher K Snider; Jann Spiess; Eli Tsukayama; Lyle Ungar; Christophe Van den Bulte; Kevin G Volpp; Angela L Duckworth

doi:10.1073/pnas.2115126119

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Proc Natl Acad Sci U S A. 2022 Feb 8;119(6):e2115126119. doi: 10.1073/pnas.2115126119.

Authors

Katherine L Milkman¹, Linnea Gandhi², Mitesh S Patel³, Heather N Graci⁴, Dena M Gromet⁴, Hung Ho⁵, Joseph S Kay⁴, Timothy W Lee⁶, Jake Rothschild⁴, Jonathan E Bogard⁷, Ilana Brody⁷, Christopher F Chabris⁸, Edward Chang⁹, Gretchen B Chapman¹⁰, Jennifer E Dannals¹¹, Noah J Goldstein¹², Amir Goren⁸, Hal Hershfield¹³, Alex Hirsch¹⁴, Jillian Hmurovic¹⁵, Samantha Horn¹⁰, Dean S Karlan¹⁶, Ariella S Kristal¹⁷, Cait Lamberton¹⁸, Michelle N Meyer⁸, Allison H Oakes¹⁹, Maurice E Schweitzer², Maheen Shermohammed⁸, Joachim Talloen¹⁰, Caleb Warren²⁰, Ashley Whillans⁹, Kuldeep N Yadav²¹, Julian J Zlatev⁹, Ron Berman¹⁸, Chalanda N Evans²², Rahul Ladhania²³, Jens Ludwig²⁴, Nina Mazar²⁵, Sendhil Mullainathan²⁶, Christopher K Snider²⁷, Jann Spiess²⁸, Eli Tsukayama²⁹, Lyle Ungar¹⁴, Christophe Van den Bulte¹⁸, Kevin G Volpp³⁰, Angela L Duckworth^{2

31}

Affiliations

¹ Department of Operations, Information and Decisions, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104; kmilkman@wharton.upenn.edu.
² Department of Operations, Information and Decisions, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
³ Ascension Health, St. Louis, MO 63105.
⁴ Behavior Change for Good Initiative, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
⁵ Department of Marketing, The University of Chicago Booth School of Business, Chicago, IL 60637.
⁶ School of Professional Studies, Northwestern University, Evanston, IL 60208.
⁷ Department of Behavioral Decision Making, Anderson School of Mangement, University of California, Los Angeles, CA 90095.
⁸ Behavioral Insights Team, Steele Institute for Health Innovation, Geisinger Health System, Danville, PA 17822.
⁹ Negotiation, Organizations & Markets Unit, Harvard Business School, Harvard University, Boston, MA 02163.
¹⁰ Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, PA 15213.
¹¹ Department of Strategy and Management, Tuck Business School, Dartmouth College, Hanover, NH 15213.
¹² Department of Management and Organizations, Anderson School of Management, University of California, Los Angeles, CA 90095.
¹³ Department of Marketing, Anderson School of Management, University of California, Los Angeles, CA 90095.
¹⁴ Department of Computer and Information Science, School of Engineering, University of Pennsylvania, Philadelphia, PA 19104.
¹⁵ Wharton Risk Management and Decision Processes Center, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
¹⁶ Department of Finance, Kellogg School of Management, Northwestern University, Evanston, IL 60208.
¹⁷ Department of Organizational Behavior, Harvard Business School, Boston, MA 02163.
¹⁸ Department of Marketing, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104.
¹⁹ Anthem, Inc., Indianapolis, IN 46203.
²⁰ Department of Marketing, Eller College of Management, University of Arizona, Tucson, AZ 85721.
²¹ Department of Biostatistics, Epidemiology, & Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104.
²² Penn Medicine Nudge Unit, Penn Medicine, Philadelphia, PA 19104.
²³ Department of Health Management and Policy, University of Michigan School of Public Health, Ann Arbor, MI 48109.
²⁴ Harris School of Public Policy, University of Chicago, Chicago, IL 60627.
²⁵ Department of Marketing, Questrom School of Business, Boston University, Boston, MA 02215.
²⁶ Department of Behavioral Science, Booth School of Business, University of Chicago, Chicago, IL 60637.
²⁷ Center for Health Care Innovation, Penn Medicine, Philadelphia, PA 19104.
²⁸ Department of Operations, Information, and Technology, Graduate School of Business, Stanford University, Stanford, CA 94305.
²⁹ Business Administration Division, University of Hawai`i-West O`ahu, Kapolei, HI 96707.
³⁰ Penn Center for Health Incentives and Behavioral Economics, Departments of Medical Ethics and Health Policy and Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104.
³¹ Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104.

Abstract

Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was "waiting for you." Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

Keywords: COVID-19; field experiment; influenza; nudge; vaccination.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Aged
COVID-19
Female
Humans
Immunization Programs*
Influenza Vaccines / administration & dosage*
Influenza, Human / prevention & control
Male
Middle Aged
Pharmacies* / statistics & numerical data
Reminder Systems
Text Messaging
Vaccination / methods*
Vaccination / statistics & numerical data

Substances

Influenza Vaccines