This paper is in the following e-collection/theme issue:

Geriatrics (24) Crowdsourcing and Mechanical Turks (98) Pharmacology and Pharmaceutics (66) Safety and Error Prevention in Health (202) Health Services Research and Health Care Utilization in Older Patients (86)

Published on in Vol 13 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/58635, first published .
Perception of Medication Safety–Related Behaviors Among Different Age Groups: Web-Based Cross-Sectional Study

Perception of Medication Safety–Related Behaviors Among Different Age Groups: Web-Based Cross-Sectional Study

Perception of Medication Safety–Related Behaviors Among Different Age Groups: Web-Based Cross-Sectional Study

Authors of this article:

Yan Lang1 Author Orcid Image ;   Kay-Yut Chen2 Author Orcid Image ;   Yuan Zhou3 Author Orcid Image ;   Ludmila Kosmari4 Author Orcid Image ;   Kathryn Daniel4 Author Orcid Image ;   Ayse Gurses5 Author Orcid Image ;   Richard Young6 Author Orcid Image ;   Alicia Arbaje7 Author Orcid Image ;   Yan Xiao4 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    1Department of Business, State University of New York at Oneonta, Oneonta, NY, United States

    2College of Business, University of Texas at Arlington, Arlington, TX, United States

    3Department of Industrial, Manufacturing, and Systems Engineering, University of Texas at Arlington, Arlington, TX, United States

    4College of Nursing and Health Innovation, University of Texas at Arlington, Arlington, TX, United States

    5School of Medicine, Johns Hopkins University, Baltimore, MD, United States

    6Family Medicine Residency Program, The John Peter Smith (JPS) Health Network, Fort Worth, TX, United States

    7Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States

    †deceased

    Corresponding Author:

    Yan Lang, PhD

    Department of Business

    State University of New York at Oneonta

    108 Ravine Pkwy

    Oneonta, NY, 13820

    United States

    Phone: 1 607 436 3251

    Email: yan.lang@oneonta.edu


    Background: Previous research and safety advocacy groups have proposed various behaviors for older adults to actively engage in medication safety. However, little is known about how older adults perceive the importance and reasonableness of these behaviors in ambulatory settings.

    Objective: This study aimed to assess older adults’ perceptions of the importance and reasonableness of 8 medication safety behaviors in ambulatory settings and compare their responses with those of younger adults.

    Methods: We conducted a survey of 1222 adults in the United States using crowdsourcing to evaluate patient behaviors that may enhance medication safety in community settings. A total of 8 safety behaviors were identified based on the literature, such as bringing medications to office visits, confirming medications at home, managing medication refills, using patient portals, organizing medications, checking medications, getting help, and knowing medications. Respondents were asked about their perception of the importance and reasonableness of these behaviors on a 5-point Likert rating scale in the context of collaboration with primary care providers. We assessed the relative ranking of behaviors in terms of importance and reasonableness and examined the association between these dimensions across age groups using statistical tests.

    Results: Of 1222 adult participants, 125 (10.2%) were aged 65 years or older. Most participants were White, college-educated, and had chronic conditions. Older adults rated all 8 behaviors significantly higher in both importance and reasonableness than did younger adults (P<.001 for combined behaviors). Confirming medications ranked highest in importance (mean score=3.78) for both age groups while knowing medications ranked highest in reasonableness (mean score=3.68). Using patient portals was ranked lowest in importance (mean score=3.53) and reasonableness (mean score=3.49). There was a significant correlation between the perceived importance and reasonableness of the identified behaviors, with coefficients ranging from 0.436 to 0.543 (all P<.001).

    Conclusions: Older adults perceived the identified safety behaviors as more important and reasonable than younger adults. However, both age groups considered a behavior highly recommended by professionals as the least important and reasonable. Patient engagement strategies, common and specific to age groups, should be considered to improve medication safety in ambulatory settings.

    Interact J Med Res 2024;13:e58635

    doi:10.2196/58635

    Keywords

    medication safety; patient engagement; aged adults; survey; Amazon Mechanical Turk; medication; engagement; older adults; elderly; safety; United States; USA; crowdsourcing; community; patient portal; primary care; medications; safety behavior; younger adults; age; correlation; statistical test 


    Engagement of older adult patients has been recognized as key to health outcomes including safety [1]. While specific skills and attitudes related to patient engagement have been identified and measured [2,3], there is a lack of clarity regarding the specific roles and responsibilities expected of patients in community settings, where patients and families are responsible for the medication use process. Health care organizations often set implicit expectations regarding the roles and responsibilities of patients and their families in collaborative activities such as planning, implementation, and discourse about their health [4].

    The purpose of our study was to use a crowdsourcing approach to investigate individuals’ perceptions of the importance and reasonableness of medication safety behaviors across various age groups. We chose patient portal use as a reference for patient engagement behaviors due to extensive efforts by health care organizations and regulators to encourage this behavior. A survey conducted in 2020 showed that more than half of individuals nationwide were offered access to patient portals, with nearly 40% accessing their records [5]. Patients’ perspectives on the importance and reasonableness of using portals are important to understand in order to devise interventions to encourage the behavior, such as patients’ interest, willingness, and ability [6-8], especially among older adults [9].

    Health care professionals are encouraged to guide patients and their families to actively participate in their care by adopting safety behaviors, although little is known about older adults’ perspectives on these behaviors and roles within the collaborative process to improve patient engagement in medication safety. By understanding the perspectives of laypeople, we can identify the gaps in engaging patients and family members in medication safety improvements and design interventions that can better meet their needs. Furthermore, understanding the relation between importance and reasonableness in perceiving medication safety behaviors is crucial for health care professionals to tailor their interventions and communication strategies effectively, particularly for older adults, to promote safer medication practices.

    The aim of this study is to assess how older adults perceive the importance and reasonableness of 8 medication safety behaviors in ambulatory settings and to compare their responses with those of younger adults.


    Study Design

    This cross-sectional study was conducted using a role-playing survey to assess the importance and reasonableness of medication safety behaviors.

    Setting

    The study was conducted using Amazon Mechanical Turk (MTurk), a crowdsourcing platform, from October to December 2022. Participants completed the survey online. This approach allowed us to efficiently gather data from a large group of participants [10].

    Participants

    Our study limited participants to US adults (≥18 years old) who had established a strong reputation on MTurk, defined as completed 100+ tasks with at least 95% approval ratings [11]. This choice leverages the acknowledged representativeness of US MTurk samples for diverse psychological dimensions [12] while ensuring engaged and reliable participation, as users with good reputations are generally more motivated and provide accurate data [13]. We used the Software Platform for Human Interaction Experiments (SoPHIE; SoPHIELabs) to administer the surveys. Participants were screened for eligibility through SoPHIE and were required to read and sign a consent form before participating. Qualified participants were given an online consent form, where they expressed their voluntary agreement to participate by clicking on the “Continue” button on their computer screen.

    Assessments

    Participants were asked to envision themselves as older adults, retired individuals living alone with multiple health conditions (detailed instructions in Figure S1 in Multimedia Appendix 1). To identify safety behaviors in managing medication use in ambulatory settings, we reviewed literature and recommendations from safety organizations to represent professionals’ views on what patients should do to contribute to medication safety. For example, 1 study targeted behaviors associated with an office visit for patient engagement, including writing out a list of medications or bringing medications to visit [14]. In our survey, we defined the “importance” of a behavior as the extent to which all patients and families should adopt it for medication safety. “Reasonableness” was judged based on the assumption that following a treatment regimen makes sense if it leads to better health outcomes [15].

    Study Survey

    This study used a carefully developed survey instrument. Initial pilot studies, with 14 closed-ended questions, assessed medication safety behaviors. Based on feedback from participants and experts, the survey was refined and consolidated for clarity and focus, resulting in a final 8-item instrument (Table 1). Throughout pilot testing, we iteratively evaluated the content validity of the questions against existing literature and organizational recommendations in medication safety. While patient involvement in developing the criteria and indicators was not direct, they were informed by a comprehensive review of relevant literature, safety advocacy group recommendations, and expert consensus in the field. Safe self-administration of medication heavily relies on patients’ knowledge about their treatments [16], their purpose, proper usage instructions, identifying and reporting adverse effects, obtaining refills, and effectively communicating any issues related to their prescribed medications with their health care provider. While the 8 behaviors specified in this study are not exhaustive, they encompass these crucial components and are presented for ease of understanding and application.

    Table 1. Targeted patient behaviors in medication safety used in the survey.
    Medication safety behaviorsExamples provided to participantsJustifications and references
    Bring medications

    		Patients are expected to bring all medications and all relevant health-related documents to their health care provider office visits.Collect all medicine bottles, including those over the counter such as Tylenol and vitamins, and bring them with you to the health care provider’s office. Make sure to also bring documents such as medication lists and blood sugar and blood pressure logs (if asked to keep one).
    • The FDAa recommends keeping a list of all medications (prescribed and over the counter) and bringing it to all doctors’ appointments [17].
    • Although often encouraged by primary care providers [18,19].
    • Only about 20%-40% of the patients bring in their medications or medication list.
    • Medication reviews lack standardization which can result in increased mortality, morbidity, and poor patient outcomes [20,21].
    Confirm medications

    		Patients will verify any changes in their medications after each provider’s visit.You have been taking 20 mg of Simvastatin every day for cholesterol for a long time. In the last visit, your cholesterol level has decreased. Your provider reduced the medication dose to 10 mg. You make this change on your personal medication list.
    • FDA recommends to verify the medication list at least once a year or any time there is a change [17].
    • Medication discrepancies are very common among patients with chronic conditions, especially those who require frequent hospitalizations or see numerous providers [22,23].
    Refill system

    		Patients will establish a refill system.Your provider advised you to call the pharmacy when you are about to run out of refills, and not their office. The pharmacy will contact the provider’s office for refill prescriptions. Using two 7-day pill boxes allows you to know 2 weeks in advance when a medicine will run out.
    • Current recommendations are to address all refill needs during the provider’s visit and send all prescriptions ideally to one pharmacy only in order to prevent gaps in medication therapies in chronic care due to disruptions and lapses in obtaining refills timely. Innovative systems use technological advances; however, older patients and individuals who speak English as a second language are less likely to use technology to refill medications [24-28].
    Use portals

    		Using patient health care portals.Your provider’s office sent you a link for creating an account to access the patient portal website. After you sign up, you can use the portal to communicate with your provider and access your health information.
    • Patient portals were intended to improve the communication between the health care team and patients. They allow patients to be actively involved in their care, access their medical records, verify for accuracy, report concerns, and seek medical advice or medication refill [6,8,29-31].
    Organize medications

    		Using pill dispensers and other organizer tools.Pill boxes are effective tools to remind you when and what medicines to take. You may also set reminders on your phone. To-go boxes are convenient to carry in your bag or purse when you are out, running errands. Charts, calendars, and electronic pill boxes are other ways of organizing medications.
    • The importance of having a system to organize medications was extensively studied. A list of memory tips and reminder systems (such as daily pillbox, calendar, or chart) to help organize scheduled prescriptions are available on various online resources [32-35].
    Check medications

    		Verifying medications for duplicates and expired medications.The mail-order pharmacy sends you your refills automatically, so they always arrive before you run out of it. You know how to check the medicines against your list, as the color of the pills and names of medications (eg, generic vs brand name) may change from time to time, and you do not want to take duplicate medicines. You also dispose of expired medicines, so you do not accidentally take them.
    • The FDA and NIHb recommend that patients check all medications for expiration dates. Ingesting expired medications may pose significant health hazards [17,36].
    Medication awareness

    		Accessing resources pertaining to medication-related issues.You went to the pharmacy to pick up a prescription, but they did not have it. To clarify the situation, you call your provider’s office to inquire if you are still supposed to take the medication and verify the correct pharmacy on file.
    In another situation, you may need to call the pharmacist or the provider to find out what you need to do if you accidentally doubled your heart medicine.
    • The World Health Organization recommends that patients learn to identify and report any issues or side effects pertaining to taking medication [17,36].
    Know medications

    		Have basic knowledge about medications.When you look at your medicine bottle, you are able to locate the medicine name, dose, when and how to take this medicine, how many refills are left, expiration date, and telephone number to call if you have questions about this medicine. For instance, you are prescribed to take a round white pill twice a day for high blood pressure. You wrote the name (“metoprolol”) on the medication list. You know to take 1 pill in the morning and 1 pill in the evening. You also know that you should take the pill with food. Symptoms to watch for are lightheadedness or very slow heartbeats.
    • Patients are advised to read carefully all information provided with the medications such as package inserts and pharmacy instructions. As many as an estimated 87% of patients do not read these instructions. Health illiteracy continues to be a challenge. Patients who are younger and have a higher formal education are more likely to have an adequate knowledge of medications [35,37-39].

    aFDA: US Food and Drug Administration.

    bNIH: National Institutes of Health.

    Data Sources

    Participants were asked to rate their perceptions of these behaviors in terms of importance and reasonableness on a 5-point Likert scale, with response categories of 1 (strongly disagree), 2 (disagree), 3 (neither agree nor disagree), 4 (agree), and 5 (strongly agree). After completing the survey, participants were asked to provide their demographic information (age, sex, race, ethnicity, education, and income) and number of chronic medical conditions.

    Study Size

    The study size was determined by targeting a minimum of 1000 participants to ensure sufficient power to detect differences between age groups. This target was based on previous studies in the field that used similar methodologies and sample sizes to achieve robust statistical power and generalizability of findings [40,41]. We received 1222 completed surveys, achieving a completion rate of 94.5%, of the total 1293 attempts.

    Data Analysis

    Statistical analyses were performed using Stata software (version 17; StataCorp). Differences in importance and reasonableness between 2 age groups (younger than 65 years and 65 years or older) were assessed by the Wilcoxon rank-sum tests. The associations between importance and reasonableness were assessed by the Pearson correlation coefficient. Disagreement was assessed by differences in ratings of reasonableness and importance scores for the same behavior by the same participant, with serious disagreement defined as a difference of 3 or more points. For ordered logistic regression models, the outcome variables were the perceived importance and reasonableness of the 8 medication safety behaviors, while the independent variables included age group, sex, ethnicity, education, income, and the number of chronic medical conditions. Each model used only 1 independent variable at a time to assess its individual impact on the outcome. Subgroup analyses with ordered logistic regression based on sex, ethnicity, education, income, and chronic medical conditions were conducted to understand how various factors influenced age-related perceptions of the 8 health behaviors.

    Ethical Considerations

    Human Participant Ethics Review Approvals or Exemptions

    This study was approved by the institutional review board (IRB) at the University of Texas at Arlington (protocol 2022-0581). The research involved human participants and adhered to appropriate ethical review and approvals as per institutional guidelines.

    Informed Consent

    Participants were given an online consent form, detailing the study’s purpose, procedures, potential risks, and benefits. They expressed their voluntary agreement to participate by clicking on the “Continue” button on their computer screen. Participants were informed of their ability to opt out of the study at any time without any consequences.

    Privacy and Confidentiality

    All data collected were anonymized to protect the privacy and confidentiality of participants. No personally identifiable information was collected. Data were stored securely on password-protected servers, and only the research team had access to the anonymized data set.

    Compensation Details

    Participants were compensated for their time and effort. Each participant received US $0.25 for completing the survey, which is a standard compensation rate for similar studies conducted on MTurk. This compensation was designed to be fair and transparent, ensuring that participants were adequately reimbursed for their contribution to the research.


    We received 1222 completed surveys (completion rate of 94.5%) from a total of 1293 attempts in December 2022. Participants took an average of 3 minutes to complete the survey. The majority of the participants were younger than 65 years, White, held a bachelor’s degree, reported an income range of US $40,000-80,000, and had 1 or more chronic conditions (Table 2). Detailed comparisons on sex, race, education, income, and number of chronic conditions are reported in the Multimedia Appendix 1.

    Across age groups, “confirming medications” was rated as the most important behavior while “knowing medications” was rated as the most reasonable behavior (Figure 1). In contrast, the behavior of using portals received the lowest scores for both importance and reasonableness. The perceived importance and reasonableness of each behavior were positively correlated (P<.001; correlations in Table S1 in Multimedia Appendix 1). Serious disagreements between importance and reasonableness for the 8 behaviors were between 6% and 7.3% among the participants (Table S2 in Multimedia Appendix 1). In addition, older adult participants reported higher importance and lower reasonableness ratings across the 8 identified behaviors (Multimedia Appendix 2).

    Among older adult participants, the behavior of confirming medication was scored highest in importance, whereas the younger age participants scored the behavior of bringing medications as highest in importance (Multimedia Appendix 3). Furthermore, among the older adult participants, the behaviors of getting help (P=.007) and knowing medications (P=.03) were rated as the second and third most important, respectively, and were found to be significantly higher than those rated by younger adult participants.

    The behavior of using portals scored lowest in both reasonableness and importance for both age groups. In addition, there was no significant difference in the scoring of this behavior between the 2 age groups. Out of the total, 3 of the remaining behaviors were scored significantly higher by older adult participants than younger adult participants (Multimedia Appendix 3).

    Across different age groups, we observed a general negative association between the perceived importance and reasonableness of these behaviors with male, Hispanic or Latino ethnicity, and education levels. Conversely, higher income was associated with a positive perception of importance. In addition, chronic medical conditions were linked to a negative perception of reasonableness. A detailed breakdown of these findings by individual behavior is shown in the Multimedia Appendix 1.

    Table 2. Demographics of study participants (n=1222).
    Characteristic<65 years≥65 yearsProportion test across age groups, P value
    Age (years), mean (SD)36.77 (10.73)68.70 (2.95)—a
    Female sex, n (%)497 (45.31)64 (51.2).21
    Race, n (%)

    		White934 (85.1)83 (66.4)<.001

    		Non-White163 (14.9)42 (33.6)<.001
    Hispanic or Latino ethnicity, n (%)327 (29.8)40 (32).61
    Education, n (%)

    		Less than bachelor’s degree131 (11.9)42 (33.6)<.001

    		Bachelor’s degree679 (61.9)45 (36)<.001

    		Graduate degree287 (26.2)38 (30.4).31
    Annual household income (US $), n (%)

    		<40,000300 (27.3)44 (35.2).07

    		40,000-80,000659 (60.1)70 (56).38

    		>80,000138 (12.6)11 (8.8).22
    Chronic medical conditions, n (%)

    		0418 (38.1)24 (19.2)<.001

    		≥1679 (61.9)101 (80.8)<.001

    a—: not applicable.

    ‎
    Figure 1. Perceptions of behaviors in terms of importance and reasonableness (n=1222). Range scale was used.

    Our study found that older adults perceive a higher importance and reasonableness of medication safety behaviors compared with younger adults. Specifically, confirming medications and knowing medications were rated as the most important and reasonable behaviors across age groups while using patient portals was perceived as the least important and reasonable.

    Previous research and safety advocacy organizations have suggested behaviors that patients can adopt to improve medication safety in ambulatory settings. Figure 1 provides an overview of participants’ perceptions of each behavior in terms of importance and reasonableness. Despite advocacy from policy makers and professional organizations for patient portal usage [6,42], the results show that it is perceived as the least reasonable and important behavior across age groups. This could be due to a perceived disconnection between portals and medication-related safety, although our study did not directly explore this link. It is important to note that the survey question about patient portal usage did not explicitly link its use to performing other medication safety behaviors, which may have influenced participants’ perceptions. Nonetheless, the discrepancy highlights the importance of understanding patients’ perspectives in shaping policy and clinical practices.

    Conversely, the behaviors of confirming medications and knowing medications received the highest scores in terms of both importance and reasonableness. These findings emphasize the significance of these behaviors in patient engagement and medication safety efforts. Health care providers should recognize the value placed by patients on these aspects of medication management and incorporate discussions and interventions related to these behaviors into clinical practice.

    Our analysis also revealed a strong correlation between reasonableness and importance for all behaviors among participants. This suggests that intervention strategies may consider targeting efforts to explain the importance and values of these behaviors. This aligns with a cost-benefit thinking approach, where something is considered reasonable when its benefits (ie, importance) are perceived as higher than the efforts required [43]. By emphasizing the benefits of engaging in medication safety practices, health care providers can encourage patients to adopt these behaviors more effectively.

    Our results suggested that older adults may be more cognizant of and experienced with health issues, making them more willing to expend effort to carry out the identified behaviors. Further research is needed to understand the reasons behind this observed difference in scores between the 2 age groups.

    Our study has several limitations. While role-playing experiments are widely used in marketing science and health care [10,44], the online platform used in the study, MTurk, may introduce biases in the sample population [45]. Consequently, the perspectives of our participants may not fully represent the broader demographic, particularly older adults, in terms of sex, race, education level, and chronic conditions. Furthermore, the underrepresentation of participants older than 65 years (10% vs 17% in the US population) could limit the generalizability of our findings. However, we mitigated this by analyzing participants older than 65 years separately from those younger than 65 years, allowing us to explore differences in perceptions of medication safety behaviors between these distinct age groups.

    In addition, it is important to recognize the complexity inherent in assessing perceptions of importance, particularly across diverse demographic groups. While our study aimed to capture patient perspectives on medication safety behaviors, the construct of importance is multifaceted and may be influenced by individual experiences, beliefs, and priorities. This highlights the potential for bias in responses when using a single age frame, particularly for younger participants who might underestimate the capabilities and perspectives of older adults.

    Furthermore, the survey question about patient portal usage did not explicitly link its use to performing other medication safety behaviors, which may have influenced participants’ perceptions. The lack of context regarding the comprehensive use of patient portals could have impacted the ratings given by participants, thus presenting a limitation in accurately assessing the perceived importance and reasonableness of using patient portals.

    Future research should consider using alternative sampling methods, such as stratified sampling or oversampling of underrepresented groups, to ensure greater representativeness in the sample. Conducting similar studies with a purposive sampling strategy could provide a more comprehensive understanding of patient perceptions of medication safety behaviors across diverse demographic groups, including older adults with multiple chronic conditions. Given the study’s limited scope, qualitative components were not integrated into the survey. Subsequent studies should encompass qualitative interviews with a spectrum of individuals, both laypersons and professionals. Furthermore, careful consideration should be given to the age-related framing of survey questions. Using age-specific framing tailored to different groups, such as presenting scenarios relevant to their age experiences, could minimize bias and provide more accurate insights into participant perceptions and behaviors across various age ranges, ultimately leading to findings with greater validity and generalizability.

    Conclusion

    Our study found that older adults perceive higher importance of a set of safety medication behaviors and see these behaviors as more reasonable to perform than younger adults. Using portals is generally perceived lower in importance and reasonableness by patients in ambulatory settings when compared with other medication safety behaviors, such as bringing medications to clinic visits. Future studies should explore additional factors influencing patient engagement in medication safety behaviors including social determinants of health. Longitudinal studies are needed to understand how improvement efforts can take advantage of patient perspectives on medication safety to design interventions to encourage the adoption of specific behaviors.

    Acknowledgments

    This project was supported by a grant (R18HS027277) from the Agency for Healthcare Research and Quality (AHRQ). The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ. We benefited from the contribution of the PROMIS (Partnership for Resilience in Medication Safety) Lab members, including Noah Hendrix, Kimbley Fulda, Anna Espinoza, Kathleen Sutcliffe, and Samantha Pitts.

    Data Availability

    All data generated or analyzed during this study are included in this published paper and its supplementary information files.

    Conflicts of Interest

    As per the University of Texas at Arlington policy, the following statement is included. K-YC has a potential research conflict of interest due to a financial interest with companies Hewlett-Packard Enterprise, Boostr, and DecisionNext. A management plan has been created to preserve objectivity in research in accordance with the University of Texas at Arlington policy. All other authors had neither competing financial interests nor other potential conflicts of interest.

    Multimedia Appendix 1

    Mturk Experimental Instructions.

    DOCX File , 304 KB
    Multimedia Appendix 2

    Perceptions of combined behaviors in terms of importance and reasonableness among age groups (n=1097 for age <65 years and n=125 for age ≥65 years).

    DOCX File , 73 KB
    Multimedia Appendix 3

    Perceptions of behaviors in terms of importance and reasonableness among age groups (n=1097 for age <65 years and n=125 for age ≥65 years). Wilcoxon rank-sum test P<.001.

    DOCX File , 177 KB
    1. Sharma AE, Rivadeneira NA, Barr-Walker J, Stern RJ, Johnson AK, Sarkar U. Patient engagement in health care safety: an overview of mixed-quality evidence. Health Aff (Millwood). 2018;37(11):1813-1820. [FREE Full text] [CrossRef] [Medline]
    2. Hibbard JH, Collins PA, Mahoney E, Baker LH. The development and testing of a measure assessing clinician beliefs about patient self-management. Health Expect. 2010;13(1):65-72. [FREE Full text] [CrossRef] [Medline]
    3. Kim JM, Suarez-Cuervo C, Berger Z, Lee J, Gayleard J, Rosenberg C, et al. Evaluation of patient and family engagement strategies to improve medication safety. Patient. 2018;11(2):193-206. [CrossRef] [Medline]
    4. Batalden M, Batalden P, Margolis P, Seid M, Armstrong G, Opipari-Arrigan L, et al. Coproduction of healthcare service. BMJ Qual Saf. 2016;25(7):509-517. [FREE Full text] [CrossRef] [Medline]
    5. Health Information National Trends Survey 5 (HINTS 5), Cycle 4 Methodology Report. Westat URL: https://hints.cancer.gov/docs/methodologyreports/HINTS5_Cycle4_MethodologyReport.pdf [accessed 2023-06-18]
    6. Neuner J, Fedders M, Caravella M, Bradford L, Schapira M. Meaningful use and the patient portal: patient enrollment, use, and satisfaction with patient portals at a later-adopting center. Am J Med Qual. 2015;30(2):105-113. [FREE Full text] [CrossRef] [Medline]
    7. Yamin CK, Emani S, Williams DH, Lipsitz SR, Karson AS, Wald JS, et al. The digital divide in adoption and use of a personal health record. Arch Intern Med. 2011;171(6):568-574. [CrossRef] [Medline]
    8. Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res. 2015;17(6):e148. [FREE Full text] [CrossRef] [Medline]
    9. Zoorob D, Hasbini Y, Chen K, Wangia-Anderson V, Moussa H, Miller B, et al. Ageism in healthcare technology: the older patients' aspirations for improved online accessibility. JAMIA Open. 2022;5(3):ooac061. [FREE Full text] [CrossRef] [Medline]
    10. Chen K, Lang Y, Zhou Y, Kosmari L, Daniel K, Gurses A, et al. Assessing interventions on crowdsourcing platforms to nudge patients for engagement behaviors in primary care settings: randomized controlled trial. J Med Internet Res. 2023;25:e41431. [FREE Full text] [CrossRef] [Medline]
    11. Chen K, Wang J, Lang Y. Coping with digital extortion: an experimental study of benefit appeals and normative appeals. Manag Sci. 2022;68(7):5269-5286. [CrossRef]
    12. O'Connell K, Berluti K, Rhoads S, Marsh A. Reduced social distancing early in the COVID-19 pandemic is associated with antisocial behaviors in an online United States sample. PLoS One. 2021;16(1):e0244974. [FREE Full text] [CrossRef] [Medline]
    13. Arch JJ, Carr AL. Using mechanical turk for research on cancer survivors. Psychooncology. 2017;26(10):1593-1603. [CrossRef] [Medline]
    14. Tennstedt SL. Empowering older patients to communicate more effectively in the medical encounter. Clin Geriatr Med. 2000;16(1):61-70, ix. [CrossRef] [Medline]
    15. Kini V, Ho PM. Interventions to improve medication adherence: a review. JAMA. 2018;320(23):2461-2473. [CrossRef] [Medline]
    16. Osterberg L, Blaschke T. Adherence to Medication. N Engl J Med. 2005;353(5):487-497. [CrossRef]
    17. 4 Medication safety tips for older adults. FDA. URL: https://www.fda.gov/consumers/consumer-updates/4-medication-safety-tips-older-adults [accessed 2023-05-30]
    18. Jäger C, Steinhaeuser J, Freund T, Szecsenyi J, Goetz K. Medication lists and brown bag reviews: potential positive and negative impacts on patients beliefs about their medicine. Biomed Res Int. 2015;2015:874067. [FREE Full text] [CrossRef] [Medline]
    19. Weiss BD, Brega AG, LeBlanc WG, Mabachi NM, Barnard J, Albright K, et al. Improving the effectiveness of medication review: guidance from the health literacy universal precautions toolkit. J Am Board Fam Med. 2016;29(1):18-23. [FREE Full text] [CrossRef] [Medline]
    20. Murtha E, Elder B, Faragher M. Brown bag medication review: using AHRQ's brown bag medication tool. J Nurs Care Qual. 2020;35(1):58-62. [CrossRef] [Medline]
    21. Twigg G, David T, Taylor J. An improved comprehensive medication review process to assess healthcare outcomes in a rural independent community pharmacy. Pharmacy (Basel). 2019;7(2):66. [FREE Full text] [CrossRef] [Medline]
    22. Brody AA, Gibson B, Tresner-Kirsch D, Kramer H, Thraen I, Coarr ME, et al. High prevalence of medication discrepancies between home health referrals and centers for medicare and medicaid services home health certification and plan of care and their potential to affect safety of vulnerable elderly adults. J Am Geriatr Soc. 2016;64(11):e166-e170. [FREE Full text] [CrossRef] [Medline]
    23. Caleres G, Modig S, Midlöv P, Chalmers J, Bondesson Å. Medication discrepancies in discharge summaries and associated risk factors for elderly patients with many drugs. Drugs Real World Outcomes. 2020;7(1):53-62. [FREE Full text] [CrossRef] [Medline]
    24. Belyeu BM, Klein JW, Reisch LM, Peacock S, Oster NV, Elmore JG, et al. Patients' perceptions of their doctors' notes and after-visit summaries: a mixed methods study of patients at safety-net clinics. Health Expect. 2018;21(2):485-493. [FREE Full text] [CrossRef] [Medline]
    25. Lyles CR, Gupta R, Tieu L, Fernandez A. After-visit summaries in primary care: mixed methods results from a literature review and stakeholder interviews. Fam Pract. 2019;36(2):206-213. [CrossRef] [Medline]
    26. Swedlund M, Kamnetz S, Birstler J, Trowbridge E, Arndt B, Micek M, et al. Reduction in medication refill encounters through primary care redesign workflow changes. J Ambul Care Manage. 2022;45(1):36-41. [CrossRef] [Medline]
    27. Vordenberg SE, Zikmund-Fisher BJ. Older adults' strategies for obtaining medication refills in hypothetical scenarios in the face of COVID-19 risk. J Am Pharm Assoc. 2020;60(6):915-922.e4. [FREE Full text] [CrossRef] [Medline]
    28. Moreno G, Lin EH, Chang E, Johnson RL, Berthoud H, Solomon CC, et al. Disparities in the use of internet and telephone medication refills among linguistically diverse patients. J Gen Intern Med. 2016;31(3):282-288. [FREE Full text] [CrossRef] [Medline]
    29. Dendere R, Slade C, Burton-Jones A, Sullivan C, Staib A, Janda M. Patient portals facilitating engagement with inpatient electronic medical records: a systematic review. J Med Internet Res. 2019;21(4):e12779. [FREE Full text] [CrossRef] [Medline]
    30. Laukka E, Huhtakangas M, Heponiemi T, Kujala S, Kaihlanen A, Gluschkoff K, et al. Health care professionals' experiences of patient-professional communication over patient portals: systematic review of qualitative studies. J Med Internet Res. 2020;22(12):e21623. [FREE Full text] [CrossRef] [Medline]
    31. Grossman LV, Masterson Creber RM, Benda N, Wright D, Vawdrey D, Ancker J. Interventions to increase patient portal use in vulnerable populations: a systematic review. J Am Med Inform Assoc. 2019;26(8-9):855-870. [FREE Full text] [CrossRef] [Medline]
    32. Medication safety tips for seniors. Oklahoma State Department of Health. 2020. URL: https:/​/oklahoma.​gov/​content/​dam/​ok/​en/​health/​health2/​aem-documents/​prevention-and-preparedness/​injury-prevention/​falls-prevention/​up-medication-safety-fact-sheet-2020.​pdf [accessed 2023-05-30]
    33. Defanti e Souza F, da Silva Santana C. A descriptive study about the use of pillboxes by older adults. Health. 2013;05(12):103-109.
    34. Mehdinia A, Loripoor M, Dehghan M, Heidari S. The effect of pillbox use on medication adherence among elderly patients: a randomized controlled trial. Int Electron J Med. 2020;9(1):38-43. [CrossRef]
    35. Cross AJ, Elliott RA, Petrie K, Kuruvilla L, George J. Interventions for improving medication-taking ability and adherence in older adults prescribed multiple medications. Cochrane Database Syst Rev. 2020;5(5):CD012419. [FREE Full text] [CrossRef] [Medline]
    36. National institute on aging. Taking Medicines Safely as You Age. URL: https://www.nia.nih.gov/health/taking-medicines-safely-you-age [accessed 2023-05-30]
    37. Svensk J, McIntyre SE. Using QR code technology to reduce self-administered medication errors. J Pharm Pract. 2021;34(4):587-591. [CrossRef] [Medline]
    38. Carvalho TPD, Okuno MFP, Campanharo CRV, Lopes MCBT, Batista REA. Patients' knowledge about medication prescription in the emergency service. Rev Bras Enferm. 2018;71(2):329-335. [FREE Full text] [CrossRef] [Medline]
    39. Kim JM, Suarez-Cuervo C, Berger Z, Lee J, Gayleard J, Rosenberg C, et al. Evaluation of patient and family engagement strategies to improve medication safety. Patient. 2018;11(2):193-206. [CrossRef] [Medline]
    40. Lee M, You M. Effects of COVID-19 emergency alert text messages on practicing preventive behaviors: cross-sectional web-based survey in South Korea. J Med Internet Res. 2021;23(2):e24165. [FREE Full text] [CrossRef] [Medline]
    41. Kim AM, Bae J, Kang S, Kim Y, Lee J. Patient factors that affect trust in physicians: a cross-sectional study. BMC Fam Pract. 2018;19(1):187. [FREE Full text] [CrossRef] [Medline]
    42. Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res. 2015;17(2):e40. [FREE Full text] [CrossRef] [Medline]
    43. Donohue K, Katok E, Leider S. The Handbook of Behavioral Operations. New Jersey, United States. John Wiley & Sons, Inc; 2018.
    44. Lang Y, Chen KY, Wu Y. Learning under the inventory problem of economic order quantity: a behavioural study. Int J Inventory Res. 2023;1(1):1. [CrossRef]
    45. Walters K, Christakis DA, Wright DR. Are mechanical turk worker samples representative of health status and health behaviors in the U.S.? PLoS One. 2018;13(6):e0198835. [FREE Full text] [CrossRef] [Medline]

    ‎
    AHRQ: Agency for Healthcare Research and Quality
    IRB: institutional review board
    MTurk: Amazon Mechanical Turk
    PROMIS: Partnership for Resilience in Medication Safety
    SoPHIE: Software Platform for Human Interaction Experiments

    Edited by T de Azevedo Cardoso; submitted 20.03.24; peer-reviewed by J Su, G Lisa, R Marshall; comments to author 15.04.24; revised version received 21.05.24; accepted 10.07.24; published 12.08.24.

    Copyright

    ©Yan Lang, Kay-Yut Chen, Yuan Zhou, Ludmila Kosmari, Kathryn Daniel, Ayse Gurses, Richard Young, Alicia Arbaje, Yan Xiao. Originally published in the Interactive Journal of Medical Research (https://www.i-jmr.org/), 12.08.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.i-jmr.org/, as well as this copyright and license information must be included.