Review
Abstract
Background: The underuse or overuse of knowledge products leads to waste in health care, and primary care is no exception.
Objective: This study aimed to characterize which knowledge products are frequently implemented, the implementation strategies used in primary care, and the implementation outcomes that are measured.
Methods: We performed a systematic review (SR) of SRs using the Cochrane systematic approach to include eligible SRs. The inclusion criteria were any primary care contexts, health care professionals and patients, any Effective Practice and Organization of Care implementation strategies of specified knowledge products, any comparators, and any implementation outcomes based on the Proctor framework. We searched the MEDLINE, EMBASE, CINAHL, Ovid PsycINFO, Web of Science, and Cochrane Library databases from their inception to October 2019 without any restrictions. We searched the references of the included SRs. Pairs of reviewers independently performed selection, data extraction, and methodological quality assessment by using A Measurement Tool to Assess Systematic Reviews 2. Data extraction was informed by the Effective Practice and Organization of Care taxonomy for implementation strategies and the Proctor framework for implementation outcomes. We performed a descriptive analysis and summarized the results by using a narrative synthesis.
Results: Of the 11,101 records identified, 81 (0.73%) SRs were included. Of these 81, a total of 47 (58%) SRs involved health care professionals alone. Moreover, 15 SRs had a high or moderate methodological quality. Most of them addressed 1 type of knowledge product (56/81, 69%), common clinical practice guidelines (26/56, 46%) or management, and behavioral or pharmacological health interventions (24/56, 43%). Mixed strategies were used for implementation (67/81, 83%), predominantly education-based (meetings in 60/81, 74%; materials distribution in 59/81, 73%; and academic detailing in 45/81, 56%), reminder (53/81, 36%), and audit and feedback (40/81, 49%) strategies. Education meetings (P=.13) and academic detailing (P=.11) seemed to be used more when the population was composed of health care professionals alone. Improvements in the adoption of knowledge products were the most commonly measured outcome (72/81, 89%). The evidence level was reported in 12% (10/81) of SRs on 62 outcomes (including 48 improvements in adoption), of which 16 (26%) outcomes were of moderate or high level.
Conclusions: Clinical practice guidelines and management and behavioral or pharmacological health interventions are the most commonly implemented knowledge products and are implemented through the mixed use of educational, reminder, and audit and feedback strategies. There is a need for a strong methodology for the SR of randomized controlled trials to explore their effectiveness and the entire cascade of implementation outcomes.
doi:10.2196/38419
Keywords
Introduction
Background
The effective implementation of knowledge products is essential for improving and sustaining the well-being of populations and reducing waste in health care. In 2019, health care spending represented 17.7% of the US gross domestic products [] and 11.5% of that for Canada []. However, the underuse of effective knowledge products that would be beneficial to the population, combined with the misuse or overuse of knowledge products that offer no added value or even provide more harm than benefits to populations, contribute to this lack of impact and waste [,]. Knowledge products include a wide range of health interventions or policies, programs, practices, or processes of technological, pharmacological, behavioral, or managerial nature and guidelines [,].
Given this gap between the production of knowledge products and their application in clinical practices and health policies, a growing emphasis has been placed on knowledge translation (KT) [,] and implementation strategies [-]. Implementation strategies can be understood as an actively planned and deliberately initiated set of processes, methods, techniques, activities, and resources, with the intention of translating a given knowledge product into practice within a particular setting and context [,-].
In recent years, given the many constraints on resources (human and financial) faced by most, if not all, health care systems, which have recently been made even worse by the COVID-19 pandemic [], there has been a growing urgency in regard to synthetizing what is known about effective implementation strategies [,-]. Despite these efforts, gaps in KT remain in relation to overviews of variable methodological and reporting qualities [], which sometimes lead to conflicting conclusions and make it challenging for health care stakeholders to decide which strategies are effective for the implementation of a given knowledge product. This concern has not been explicitly addressed in the existing literature.
Therefore, we planned a 3-phase project, with the ultimate goal to identify, for each category of knowledge product, the most effective implementation strategies for their uptake into health care professionals’ clinical practice. The first phase was to critically analyze the existing literature overviews to determine their strengths and weaknesses. This allowed us to highlight many methodological challenges such as the definition of eligibility criteria and literature search, the way in which data were synthesized, the methodological quality assessment of the literature reviews included, and the assessment of the evidence level. These points informed the realization of the present systematic review (SR) of SRs, which is the second phase of our project.
Objective
We sought to characterize which knowledge products are frequently implemented, the implementation strategies used in primary care, and the implementation outcomes measured.
Methods
Project Design and Registration
To optimize the identification of effective implementation strategies in the area of primary care, we conducted a 3-phase project using SR methodologies. In phase 1 (completed review), we conducted a critical analysis of the methodological strengths and weaknesses of the existing overviews. In phase 2 (current overview), we conducted an SR of SRs to characterize the most frequently implemented knowledge products, implementation strategies used, implementation outcomes measured, and reported levels of evidence in individuals or stakeholders participating in the provision of health care (referred to as health care professionals) or in health care professionals and end users (patients and clients) in the context of primary health care. In the included SRs, primary studies may either be of more robust experimental designs (randomized controlled trials [RCTs]) or less robust designs. Therefore, the effectiveness of key knowledge products and key implementation strategies was measured in a separate phase 3 (future review) using an SR of RCTs.
The protocol of the project was registered on the Open Science Framework platform on February 7, 2020 [] and then published []. The review was conducted following the Cochrane methodology [] and is reported in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [].
Eligibility Criteria
We used the population, intervention, comparison and outcomes format [] to delineate our inclusion criteria.
Population and Clinical Context
We included any person involved in health care provision, that is, health care professionals or caregivers and end users (patients). By caregivers, we mean the parents, guardians, friends of patients, community health workers, or any other nonclinician who provides health care. The empirical studies in the included reviews could concern either health care professionals or caregivers alone, or health care professionals or caregivers and patients. They were excluded from cases in which only the patients were concerned. We did not place restrictions on age, gender, or health conditions. Reviews had to cover the primary care setting [], as it is a major level of health service use. Rather than targeting the physical location of activities, we were interested in primary health care services, such as health promotion and prevention, diagnosis, and treatment of illness and injury. By primary health care services, we refer to family physicians, nurse practitioners, and pharmacists who ensure the direct provision of health care services to clients and coordinate to ensure the continuity of care to upper levels [].
Intervention
We focused on implementation strategies that were predetermined in our protocol [] and based on the Effective Practice and Organization of Care (EPOC) [] to include the following implementation strategies: audit, feedback, audit and feedback, clinical incident reporting, monitoring the performance of the delivery of health care, communities of practice, continuous quality improvement, educational games, educational materials, educational meetings, educational outreach visits or academic detailing, clinical practice guidelines, interprofessional education, local consensus processes, local opinion leaders, managerial supervision, patient-mediated interventions, public release of performance data, reminders, routine patient-reported outcome measures, and tailored interventions. A review may have included primary studies that use exclusively 1 type of implementation strategy (mono-faceted) or exclusively more than 1 type (multifaceted). Within the same review, some primary studies may have used exclusively one implementation strategy, whereas others may have used exclusively more than one implementation strategy (mixed). We excluded interventions that were used to develop the knowledge product and the scaling up and sustainability of interventions (studies that were housed under a separate project). Knowledge products are tools used to share knowledge with users [,]. They include tools such as clinical practice guidelines, decision support tools, policy briefs or decision-making tools, one pagers, and health interventions (technological, pharmacological, behavioral, or management). In the clinical practice guidelines category, we included clinical practice guidelines, disease management protocols, clinical recommendations, and clinical procedures. For health interventions, we included knowledge products for which the implementation aimed to change professional behavior or attitude (behavioral), professional competencies or processes or quality of care (management), prescribing or testing (pharmacological), and the use of technologies (technological). In the shared decision-making and support tools category, we included clinical decision support systems and tools aimed at improving clinical decision-making. In a given SR, 1 type of knowledge product (single) or more than 1 type (multiple) may have been implemented. A review was included if the knowledge product and implementation strategies were specified.
Comparators
We considered either usual practice (no predetermined implementation strategies as defined previously) or any of the predetermined implementation strategies defined earlier.
Outcomes
Our interest was focused on implementation outcomes, including acceptability, adoption, appropriateness, feasibility, adherence or fidelity, implementation costs, and penetration or reach of a knowledge product, as defined in the taxonomies by Proctor et al [] and Lewis et al []. Detailed definitions are provided in . Several of these outcomes may have been studied in the same SR.
Design of Included Reviews
We included both Cochrane and non-Cochrane SRs (with or without meta-analyses) and mixed method reviews that used a comprehensive and reproducible approach and met our inclusion criteria. The reviews may have included one or more types of experimental or observational primary study designs. We excluded reviews of reviews, non-SRs, original research, protocols, comments, editorials, conference abstracts, working groups and colloquium reports, experts’ opinions, and pilot studies.
Information Sources and Search Strategy
We searched MEDLINE (Ovid), EMBASE, CINAHL (EBSCO), Ovid PsycINFO (Ovid), Web of Science (Web of Science), and Cochrane Library (Cochrane Library) databases from their inception to October 18, 2019, without restrictions on language or geographic settings. We searched the bibliographies of the included reviews to identify additional relevant ones.
We followed an extensive literature search process to identify SRs of interventions that implement health knowledge products. In March 2017, an information specialist (NR) designed the search strategy for each database. The initial search strategy developed in MEDLINE was reviewed and approved by some of the team members before its translation into other bibliographic databases by the information specialist. During the selection process, gaps were identified in the search strategy. The search strategy was modified and rerun in October 2019. We used the following main concepts: KT, strategies, reviews, health professionals, and primary care. details the search strategy for each of the aforementioned databases. The records found were exported to the EndNote software (Clarivate), and duplicates were removed.
Study Selection
We used Microsoft Excel developed for our review to perform the study selection in 3 steps. First, our reviewers performed pilot selection and held discussions regarding any discordance to ensure a common understanding of the eligibility criteria before subsequent steps were taken. Second, pairs of reviewers independently screened the titles and abstracts. Records coded as included or unclear were eligible for a full report review against the inclusion criteria by pairs of reviewers. Third, full reports were coded on one side as included or unclear and as excluded on the other side. At each step, consensus discussions were held to resolve disagreements. A senior reviewer validated the final list of included SRs. We did not need to contact any of the review authors. A flow diagram, according to the PRISMA guidelines [], was produced to summarize the process of study selection (
Figure 1).
Data Extraction
We used the piloted Microsoft Excel format developed for our review to extract the data. To develop the format, we used the taxonomies of the EPOC [] for the categories of implementation strategies and complemented the information by specifying whether the implementation strategies were mono-faceted, multifaceted, or mixed. For the outcome definitions, we used the Proctor et al [] and Lewis et al [] evaluation frameworks. These frameworks integrate more dimensions not found in other frameworks, such as acceptability, appropriateness, feasibility, and implementation costs. They also provide outcome synonyms found in the literature, thus facilitating recategorization when needed. For each outcome, we specified whether the measurement was objective and the measurement tools used, if reported. Evidence-based interventions are practices in which health professionals use available evidence-based information to make decisions for individual patients or community health [,]. It operates by appraising evidence and formulating recommendations or guidelines [,,] and by integrating evidence and community preferences for policy and practice changes at the public health level (health interventions) [,]. We were unable to find a formal taxonomy of knowledge products; therefore, we used the literature [,,,] to categorize whether they were clinical practice guidelines, health interventions, or shared decision-making and support tools. In cases where they were health interventions, we specified their technological, pharmacological, behavioral, or management nature. Furthermore, we extracted information regarding whether the type of implemented knowledge product was single (eg, clinical practice guidelines alone) or multiple (eg, clinical practice guidelines and health interventions). The population was defined as health care professionals only or health care professionals and patients and their number and characteristics of age and gender were extracted where available.
To give context to our review, the following additional information was also extracted: general characteristics of the included review (such as year of publication, number and names of databases searched, search date ranges considered, any language restriction, method of synthesizing data, medical area of concern, settings, designs, and number of primary studies), whether the authors of the included reviews completed methodological quality assessment (tools used and overall result), whether they completed publication bias assessment (tool used and whether any treatment was done), and whether they completed the assessment of quality of evidence (tool used and level of evidence by each reported outcome).
Pairs of reviewers piloted the tool on at least 2 reviews and independently carried out extractions and validations by comparing the extracted information. Discussions for consensus were held in case of discrepancies.
Assessment of Methodological Quality
The methodological quality of the included reviews was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR; AMSTAR 2) []. In contrast to the first version, this updated version allows the assessment of SRs that include RCTs, nonrandomized studies of health interventions, or both []. We conducted a pilot phase and held discussions on discordance. Where necessary, the pilot phase was extended until a common understanding of the assessment criteria was achieved. Pairs of assessors independently scored each of these 16 items. An overall rating was also provided, which indicated high (no or one noncritical flaw), moderate (more than one noncritical flaw), low (one critical flaw with or without noncritical flaws), or critically low (more than one critical flaw with or without noncritical flaws) ratings []. Critical flaws included protocol not registered before the beginning of the review (standard 2), lack of adequacy and comprehensiveness of the search strategy (standard 4), no provision of the justification for excluding individual reviews (standard 7), the use of an unsatisfactory technique to assess the risk of bias from individual included reviews (standard 9), the inappropriateness of meta-analytical methods (standard 11), no consideration of the risk of bias when interpreting the results of the review (standard 13), and lack of suitability for the assessment of the presence and the likely impact of publication bias (standard 15) []. Reviewers compared their results and reached a consensus in cases of disagreement by discussion or by the arbitration of a third reviewer.
Data Synthesis
For the second phase, reanalysis by meta-analysis was not performed []. Using SAS software (SAS Institute Inc) and taking the included review as the unit of analysis, we performed a descriptive analysis that aimed to summarize the characteristics of the implemented knowledge products, implementation strategies used, outcomes measured, and levels of evidence reported. We summarized the data as numbers and percentages for categorical variables and as means and SDs or medians and IQRs for continuous variables. Counts were performed overall and then stratified according to methodological quality scores (high, moderate, low, and very low). We grouped Technological Health Interventions and Decision Support Tools as implemented clinical decision support tools were electronic or computerized decision support systems.
For reviews in which the level of evidence of outcomes was measured and reported, we summarized what was reported as the level of evidence for the reported implementation outcome by the implementation strategy used and by the specific implemented single knowledge product. We used the number of outcomes as the unit of analysis.
Results
Search and Selection Process
Our database search identified 11,101 records, of which 6915 (62.29%) titles and abstracts were screened after removing duplicates. Among these 6915, a total of 428 (6.19%) full reports were screened for eligibility, after which 81 (18.9%) admissible SRs remained [-] (Figure 1). The reasons for the exclusion of each examined full report are provided in .
General Characteristics of the Included Reviews and Participants
Table 1 shows the key general characteristics of the 81 included SRs. They were published between 1989 and 2019, with a mean of 8 (SD 5.8) years since the last bibliographic search in 2019. The authors of SRs searched an average of 6 databases, whereas more than half of the reviews (41/81, 51% of the SRs) restricted their search to SRs in the English language [-,,,,,,-,,,-,,-,-,,-, ,,,,,-]. Individual SRs included a mean number of 29 primary studies. Of 81 SRs, with the exception of 8 (10%) SRs [,,,,,,,], all remaining SRs (n=73, 90%) included primary studies designed as RCTs. Non-RCTs were included in 42% (34/81) of SRs [,,,,,,,,,,-,,-,,,, -,,,,,,,]. The settings covered were either primary or secondary health care (56/81, 69%) or primary health care alone in 31% (25/81) of SRs [,,,,,,,,,,,,,,,-, ,-,,]. A total of 58% (47/81) of SRs involved health care professionals alone [,,,,,,-, ,,,-,,,-,,-,-,-,,,, -,-,,,], whereas the remaining 42% (34/81) involved health care professionals and patients at the same time. In 83% (67/81) of SRs, primary studies were critically appraised for methodological quality [-,, ,-,-,-,,-,,,,,-,,], and the narrative approach was used to synthesize information in 80% (65/81) of SRs [-,,,-, ,-,-,-,-,,-,, -,-]. A detailed table of the key general characteristics for each included review is available in [-].
| Characteristics | Overall | Methodological quality | |||||||||
| High | Moderate | Low | Critically low | ||||||||
| Analyzed systematic reviews, n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Age of reviews (years), n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Value, mean (SD) | 8.0 (5.8) | 4.8 (3.1) | 9.1 (5.9) | 6.0 (3.3) | 9.1 (6.6) | ||||||
| Value, median (IQR) | 6.8 (3.8-10.8) | 3.8 (2.8-6.8) | 7.3 (5.8-8.8) | 5.8 (3.8-7.8) | 7.8 (3.8-12.8) | ||||||
| Databases searched in included reviews, n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Value, mean (SD) | 6.3 (3.8) | 10.3 (5.6) | 9.7 (4.1) | 6.6 (3.3) | 5.0 (2.8) | ||||||
| Value, median (IQR) | 5.0 (3.0) | 7.0 (10.0) | 9.5 (3.0) | 6.0 (4.0) | 5.0 (4.0) | ||||||
| Search language restriction in included reviews, n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Yes | 48 (59) | 1 (11) | 2 (33) | 12 (70) | 33 (67) | ||||||
| No | 19 (24) | 5 (56) | 4 (67) | 4 (24) | 6 (12) | ||||||
| Not reported | 14 (17) | 3 (33) | 0 (0) | 1 (6) | 10 (21) | ||||||
| Language restrictions, n (%) | 48 (100) | 1 (2) | 2 (4) | 12 (25) | 33 (69) | ||||||
| English only | 41 (85) | 1 (100) | 2 (100) | 11 (92) | 27 (82) | ||||||
| English and other languages | 7 (15) | 0 (0) | 0 (0) | 1 (8) | 6 (18) | ||||||
| Primary studies included in included reviews, n (%) | 80a (100) | 9 (11) | 6 (8) | 17 (21) | 48 (60) | ||||||
| Value, mean (SD) | 29.2 (34.7) | 20.3 (12.2) | 16.8 (13.1) | 27.2 (20.3) | 33.1 (42.3) | ||||||
| Value, median (IQR) | 19.5 (10.5-34.5) | 19.0 (12.0-26.0) | 14.0 (8.0-19.0) | 22.0 (11.0-38.0) | 19.0 (11.0-38.0) | ||||||
| How many reviews included the following designs of primary studiesb, n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Randomized controlled trials | 73 (90) | 9 (100) | 6 (100) | 15 (88) | 43 (88) | ||||||
| Nonrandomized controlled trials | 34 (42) | 2 (22) | 4 (67) | 9 (53) | 19 (39) | ||||||
| Interrupted time series | 20 (25) | 4 (44) | 3 (50) | 5 (29) | 8 (16) | ||||||
| Cohorts | 11 (17) | 1 (11) | 0 (0) | 2 (12) | 8 (16) | ||||||
| Before-after | 26 (32) | 0 (0) | 2 (33) | 5 (29) | 19 (39) | ||||||
| Other | 22 (27) | 1 (11) | 0 (0) | 5 (29) | 16 (33) | ||||||
| Settings (health domains), n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Primary and secondary health care | 56 (69) | 9 (100) | 6 (100) | 13 (76) | 28 (57) | ||||||
| Primary health care only | 25 (31) | 0 (0) | 0 (0) | 4 (24) | 21 (43) | ||||||
| Method of analysis for included reviews,n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Narrative | 65 (80) | 5 (56) | 5 (83) | 15 (88) | 40 (82) | ||||||
| Mixed synthesis | 10 (12) | 3 (33) | 1 (17) | 1 (6) | 5 (10) | ||||||
| Meta-analysis | 6 (8) | 1 (11) | 0 (0) | 1 (6) | 4 (8) | ||||||
| Were primary studies critically appraised for methodological quality, n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Yes | 67 (83) | 9 (100) | 6 (100) | 17 (100) | 35 (71) | ||||||
| No | 10 (12) | 0 (0) | 0 (0) | 0 (0) | 10 (21) | ||||||
| Not reported | 4 (5) | 0 (0) | 0 (0) | 0 (0) | 4 (8) | ||||||
| Population of reviews, n (%) | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | ||||||
| Health care professionals only | 47 (58) | 6 (67) | 5 (83) | 8 (47) | 28 (57) | ||||||
| Health care professionals and patients | 34 (42) | 3 (33) | 1 (17) | 9 (53) | 21 (43) | ||||||
aOne study without a number of included studies.
bCategories are not mutually exclusive.
Implementation Strategies, Knowledge Products, and Outcomes
Implementation Strategies
In 6% (5/81) of SRs [,,,,], all primary studies in SR used only 1 type of implementation strategy (mono-faceted). In 11% (9/81) of SRs [,,,,,,,,], all primary studies in SR exclusively used more than 1 type of implementation strategy (multifaceted). In the remaining 83% (67/81) of SRs, some primary studies used 1 type of implementation strategy, whereas others used more than 1 type of implementation strategy (mixed; Table 2). Educational strategies were the most frequently used, mainly educational meetings in 74% (60/81) of SRs [-,-,,,,,,-,-,-,-,,,-, ,,,,-,-], educational materials distribution in 73% (59/81) of SRs [-, -,-,,-,-,-,-,-,,,-,, ,,,,-,-] and educational outreach in 56% (45/81) of SRs [-,,,,-, ,,,-,,,-,,-,-,,,,,, ,-,,-,] (Table 2). Other frequent strategies used were reminders in 65% (53/81) of SRs [,,,,,,,-,-,,,-,,,,,, ,,,,,,,-,-,,,-,,], audit and feedback for 49% (40/81) of SRs [,,,,,-,,,-,,,,,,,-,,, -,,,,,,,-,,] and the use of local opinion leaders for 43% (35/81) of SRs [,,-,,,,,,,,,-,,,,,,,, ,,,,,,,,,] (Table 2).
| Characteristics | Overall, n (%) | Methodological quality, n (%) | ||||
| High | Moderate | Low | Critically low | |||
| Analyzed systematic reviews | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | |
| Type of knowledge products implemented | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | |
| Single | 56 (69) | 4 (44) | 4 (67) | 13 (76) | 35 (71) | |
| Multiple | 25 (31) | 5 (56) | 2 (33) | 4 (24) | 14 (29) | |
| Categories of single knowledge products | 56 (100) | 4 (7) | 4 (7) | 13 (23) | 35 (63) | |
| Clinical practice guidelines | 26 (46) | 0 (0) | 1 (25) | 9 (70) | 16 (46) | |
| Management, behavioral, and pharmacological health interventions | 24 (43) | 3 (75) | 3 (75) | 2 (15) | 16 (46) | |
| Health technology interventions and decision support tools | 6 (11) | 1 (25) | 0 (0) | 2 (15) | 3 (8) | |
| Types of implementation strategies | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | |
| Mixed | 67 (83) | 7 (78) | 5 (83) | 15 (88) | 40 (82) | |
| Multifaceted only | 9 (11) | 2 (22) | 1 (17) | 2 (12) | 4 (8) | |
| Mono-faceted only | 5 (6) | 0 (0) | 0 (0) | 0 (0) | 5 (10) | |
| Implementation strategies categoriesa | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | |
| Educational meetings | 60 (74) | 8 (89) | 5 (83) | 12 (71) | 35 (71) | |
| Educational materials | 59 (73) | 8 (88.9) | 6 (100) | 10 (59) | 35 (71) | |
| Reminders | 53 (64) | 8 (89) | 2 (33) | 12 (71) | 31 (63) | |
| Educational outreach visits or academic detailing | 45 (56) | 5 (56) | 5 (83) | 9 (53) | 26 (53) | |
| Audit and feedback | 40 (49) | 6 (67) | 4 (67) | 10 (59) | 20 (41) | |
| Local opinion leaders | 35 (43) | 3 (33) | 4 (67) | 7 (41) | 21 (43) | |
| Feedback | 32 (40) | 5 (56) | 1 (17) | 5 (29) | 21 (43) | |
| Clinical practice guidelines | 23 (28) | 3 (33) | 1 (17) | 4 (24) | 15 (31) | |
| Local consensus processes | 18 (21) | 2 (22) | 2 (33) | 2 (12) | 12 (25) | |
| Tailored interventions | 15 (17) | 3 (33) | 0 (0) | 2 (12) | 10 (20) | |
| Audit | 13 (16) | 1 (11) | 0 (0) | 4 (24) | 8 (16) | |
| Patient-mediated interventions | 11 (14) | 1 (11) | 0 (0) | 1 (6) | 9 (18) | |
| Interprofessional education | 9 (11) | 1 (11) | 0 (0) | 3 (18) | 5 (10) | |
| Continuous quality improvement | 9 (10) | 2 (22) | 0 (0) | 2 (12) | 5 (10) | |
| Monitoring the performance of the delivery of health care | 6 (7) | 1 (11) | 0 (0) | 1 (6) | 4 (8) | |
| Managerial supervision | 6 (7) | 0 (0) | 0 (0) | 0 (0) | 6 (12) | |
| Educational games | 5 (6) | 0 (0) | 0 (0) | 1 (6) | 4 (8) | |
| Communities of practice | 2 (3) | 0 (0) | 0 (0) | 0 (0) | 2 (4) | |
| Clinical incident reporting | 1 (1) | 0 (0) | 0 (0) | 0 (0) | 1 (2) | |
| Routine patient-reported outcome measures | 1 (1) | 1 (11) | 0 (0) | 0 (0) | 0 (0) | |
| Public release of performance data | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | |
| How outcomes were measureda | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | |
| Not reported | 47 (58) | 4 (44) | 4 (67) | 7 (41) | 32 (65) | |
| Objective | 29 (36) | 6 (67) | 4 (67) | 8 (47) | 11 (23) | |
| Both | 18 (22) | 1 (11) | 1 (17) | 4 (24) | 12 (25) | |
| Self-administered | 11 (14) | 0 (0) | 2 (33) | 1 (6) | 8 (16) | |
| Implementation outcomesa | 81 (100) | 9 (11) | 6 (7) | 17 (21) | 49 (61) | |
| Adoption | 72 (89) | 8 (89) | 6 (100) | 14 (82) | 44 (90) | |
| Other | 28 (35) | 4 (44) | 4 (67) | 4 (24) | 16 (33) | |
| Implementation costs | 16 (20) | 1 (11) | 2 (33) | 3 (18) | 10 (20) | |
| Acceptability | 15 (19) | 2 (22) | 2 (33) | 3 (18) | 8 (16) | |
| Fidelity | 9 (11) | 1 (11) | 0 (0) | 1 (6) | 7 (14) | |
| Penetration | 6 (7) | 0 (0) | 0 (0) | 0 (0) | 6 (12) | |
| Appropriateness | 5 (6) | 0 (0) | 0 (0) | 1 (5.9) | 4 (8) | |
| Sustainability | 4 (5) | 0 (0) | 0 (0) | 0 (0) | 4 (8) | |
| Feasibility | 3 (4) | 0 (0) | 0 (0) | 0 (0) | 3 (6) | |
| Other outcomesa | 28 | 4 (14) | 4 (14) | 4 (14) | 16 (58) | |
| Knowledge | 19 (68) | 1 (25) | 4 (100) | 3 (75) | 11 (69) | |
| Attitudes | 10 (36) | 2 (50) | 2 (50) | 1 (25) | 5 (31) | |
| Performance in a test situation | 9 (32) | 1 (25) | 1 (25) | 1 (25) | 6 (38) | |
| Satisfaction | 8 (29) | 2 (50) | 2 (50) | 1 (25) | 3 (19) | |
aCategories are not mutually exclusive.
Knowledge Products
Of the 81 SRs, 56 (69%) focused on the implementation of single-type knowledge products, including 26 (46%) for clinical practice guidelines [,,,-,,,-,,,, ,,,-,,], 24 (43%) for health interventions of management and behavioral or pharmacological nature [,,-,,,,,,,,-,,-, ,,,], and 6 (11%) for health technology interventions and decision support tools [,,,,,]. In the remaining 31% (25/81) of SRs, multiple knowledge products were the subjects of implementation (Table 2).
Knowledge Products by Implementation Strategy
The strategies used varied based on the knowledge product being implemented; clinical practice guidelines were commonly implemented using educational material distribution (21/26, 81%) [,,,,,,,-,,,,,,,, ,], reminders (20/26, 77%) [,,,-,, ,,,,,,,,,,,,], and academic detailing (18/26, 69%) [,,,,,,-,,, ,,,,]. The simultaneous use of these 3 strategies to implement clinical practice guidelines was reported in 60% (15/26) of SRs [,,,,,,,, ,,,,,,].
For health interventions of management and behavioral or pharmacological nature, the implementation strategies included education meetings (19/24, 79%) [,,-,, ,,,,,-,,,,,], educational material distribution (15/24, 63%) [,,,,,,,, ,-,,,], and reminders (15/24, 63%) [,, ,,,,,,-,,,,]. Their simultaneous use was reported in 42% (10/24) of SRs [,, ,,,,-,].
The same pattern was observed for health technology interventions and decision support tools implemented using education meetings (3/6, 50%) [,,], educational material distribution (3/6, 50%) [,,], and audit and feedback (3/6, 50%) [,,]. The simultaneous use of these strategies has been reported in 17% (1/6) of SRs [].
We compared the proportions of implementation strategies used when the population was health care professionals alone and when the population was health care professionals and patients. Education meetings and academic detailing seem to be used when the population is composed of health care professionals alone, without any statistical significance (Table 3).
| Implementation strategiesa | Health care professionals alone (n=47), n (%) | Health care professionals and patients (n=34), n (%) | P value |
| Education meetings | 38 (81) | 22 (65) | .13 |
| Educational materials | 37 (78) | 22 (65) | .21 |
| Academic detailing | 30 (64) | 15 (44) | .11 |
| Reminders | 30 (64) | 23 (68) | .81 |
| Audit and feedback | 26 (55) | 14 (41) | .26 |
| Local opinion leaders | 21 (45) | 14 (41) | .82 |
aCategories are not mutually exclusive.
Outcomes
The adoption of knowledge products was the commonly measured implementation outcome in 89% (72/81) of SRs [-,,,-,-,-,-,-,-,-], followed by implementation cost in 20% (16/81) of SRs [,,,,,,,,,,,,,,,], and acceptability in 19% (15/81) of SRs [,,,,,,,,,,,,,,]. Knowledge (19/81, 24%) and attitudes (10/81, 12%) were the other outcomes reported (Table 2). Further details are provided in [-].
Quality Assessment for the Included Reviews
Of the 81 included SRs, 15 (19%) received scores that indicated high or moderate methodological quality [,,,-,,,,,,]. The remaining 81% (66/81) received scores that indicated low or very low methodological quality (Table 1). The quality was lowered by 3 criteria: lack of reporting on the funding sources of the studies included in the reviews (73/81, 90%), nonprovision of the list of excluded studies (with reasons for their exclusion; 58/81,72%), and no statement on the existence of the protocol or methodology before the conduct of the review (49/81, 61%; 
Figure 2). Details for each included SR are provided in [-].
Reported Effectiveness of Implementation Strategies and Level of Evidence of Outcomes in the Included Reviews
We synthesized the evidence by combining information on the reported level of evidence for the outcomes measured, among single knowledge products implemented, and by the implementation strategy used (Table 4). For SRs in which single knowledge products were implemented (56/81, 69%), the level of evidence was reported in 10 SRs with 62 specified outcomes (Table 4). Of these, 50 outcomes were related to the implementation of clinical practice guidelines in 6 SRs [,,,,,], 5 outcomes on management and behavioral or pharmacological health interventions in 3 SRs [,,], and 7 outcomes for health technology interventions or decision support tools in one SR [] (Table 4).
Regarding clinical practice guidelines (50 outcomes with the level of evidence reported), the following implementation strategies were used: educational material distribution for 42% (21/50) of outcomes [,,,,], 71% (15/21) were about adoption [,,,,], and 27% (4/15) provided a high or moderate level of evidence [,,]; educational meetings for 30% (15/50) of outcomes [,,,], all were about adoption, 13% (2/15) provided a high or moderate level of evidence [,]; audit and feedback for 26% (13/50) of outcomes [,,,], all were about adoption, 23% (3/13) of them provided a high or moderate level of evidence [,,]; and reminders for 24% (12/50) of outcomes [,,]; all were about adoption, 33% (4/12) provided a high or moderate level of evidence [,] (Table 4).
For management and behavioral or pharmacological health interventions, educational meetings were evaluated for all 100% (5/5) of outcomes [,,], all were about adoption, and 20% (1/5) of them provided a high or moderate level of evidence []. Feedback was evaluated for 80% (4/5) of outcomes [,,]; all were about adoption, and 25% (1/4) provided a high or moderate level of evidence [] (Table 4).
Health technology interventions or decision support tools used reminders for 100% (7/7) of outcomes [], 43% (3/7) for acceptability [], with 33% (1/3) providing a high or moderate level of evidence []; furthermore, there were 29% (2/7) for implementation costs [], with 50% (1/2) providing a high or moderate level of evidence []. In addition, feedback was used for 86% (6/7) of outcomes [], 33% (2/6) for acceptability [], with 50% (1/2) providing a high or moderate level of evidence []; furthermore, there were 33% (2/6) for implementation costs [], with 50% (1/2) providing a high or moderate level of evidence [] (Table 4).
| Knowledge products, implementation strategiesa, and categories of outcomesb,c | Level of evidence | ||
| Clinical practice guidelines (n=50 outcomes) | |||
| Educational meetings (n=15) | |||
| Adoption (n=15) |
| ||
| Educational materials (n=21) | |||
| Adoption (n=15) |
| ||
| Knowledge (n=3) |
| ||
| Performance in a test situation (n=2) |
| ||
| Satisfaction (n=1) |
| ||
| Reminders (n=12) | |||
| Adoption (n=12) |
| ||
| Educational outreach visits or academic detailing (n=6) | |||
| Adoption (n=6) |
| ||
| Audit and feedback (n=13) | |||
| Adoption (n=13) |
| ||
| Local opinion leaders (n=4) | |||
| Adoption (n=4) |
| ||
| Feedback (n=12) | |||
| Adoption (n=12) |
| ||
| Clinical practice guidelines (n=3) | |||
| Adoption (n=3) |
| ||
| Local consensus processes (n=3) | |||
| Adoption (n=3) |
| ||
| Tailored interventions (n=5) | |||
| Adoption (n=5) |
| ||
| Audit (n=11) | |||
| Adoption (n=11) |
| ||
| Interprofessional education (n=3) | |||
| Adoption (n=3) |
| ||
| Continuous quality improvement (n=10) | |||
| Adoption (n=10) |
| ||
| Monitoring the performance of the delivery of health care (n=3) | |||
| Adoption (n=3) |
| ||
| Management and behavioral or pharmacological health interventions (5 outcomes) | |||
| Educational meetings (n=5) | |||
| Adoption (n=5) |
| ||
| Educational materials (n=2) | |||
| Adoption (n=2) |
| ||
| Reminders (n=2) | |||
| Adoption (n=2) |
| ||
| Educational outreach visits, or academic detailing (n=2) | |||
| Adoption (n=2) |
| ||
| Audit and feedback (n=1) | |||
| Adoption (n=1) |
| ||
| Local opinion leaders (n=1) | |||
| Adoption (n=1) |
| ||
| Feedback (n=4) | |||
| Adoption (n=4) |
| ||
| Local consensus processes (n=2) | |||
| Adoption (n=2) |
| ||
| Patient-mediated interventions (n=1) | |||
| Adoption (n=1) |
| ||
| Health technology interventions and decision support tools (7 outcomes) | |||
| Reminders (n=7) | |||
| Acceptability (n=3) |
| ||
| Adoption (n=1) |
| ||
| Fidelity (n=1) |
| ||
| Implementation costs (n=2) |
| ||
| Feedback (n=6) | |||
| Acceptability (n=2) |
| ||
| Adoption (n=1) |
| ||
| Fidelity (n=1) |
| ||
| Implementation costs (n=2) |
| ||
| Clinical practice guidelines (n=1) | |||
| Implementation costs (n=1) |
| ||
aCategories are not mutually exclusive.
bPositive outcome (eg, increase in adoption and increase in knowledge).
cWithin the same review, it may have implemented 1 type of single knowledge product (eg, clinical practice guidelines) but used different specific practices (eg, general obstetric care guidelines and emergency obstetric care guidelines). Although these practices may report the same category of implementation outcome (eg, adoption), if those practices presented and reported different levels of evidence specific for each one (eg, low for general obstetric care guidelines and moderate for emergency obstetric care guidelines), then their outcomes were extracted separately and analyzed separately.
Discussion
Principal Findings
In this paper, we report the results of an SR of SRs, thus providing a detailed portrait of (1) the knowledge products or innovations implemented in primary health care, (2) the implementation strategies used by health care professionals in primary care, and (3) implementation outcomes evaluated as well as their reported level of evidence in primary care.
The findings of this review will be used to inform future SRs of RCTs on the effectiveness of implementation strategies for specific knowledge products.
In this review, which summarized a total of 81 studies, for most (56/81, 69%) of the included SRs, only 1 type of knowledge product (single) was implemented, the majority of which were clinical practice guidelines or health interventions (of management and behavioral or pharmacological nature). Implementation strategies commonly combine education-based strategies (material distributions, meetings, and outreach), reminders, and audits and feedback. Improvement in the adoption of knowledge products was the most measured outcome.
Education-based strategies, audits, feedback, and reminders were mainly used to improve the adoption of clinical practice guidelines and health interventions related to management, behavior, or pharmacology. In contrast, reminders and audit and feedback were used to improve the acceptability and implementation costs of health technology interventions. The reported effectiveness of these strategies was of a high or moderate level of evidence in a few cases and of a low or very low level of evidence in most cases.
Comparison With Prior Work
Clinical practice guidelines and management, behavioral or pharmacological health interventions, and health technology interventions and decision support tools have been developed to improve clinical practice and patient health outcomes. Despite their comparable effectiveness, the level or degree of implementation varies widely. For instance, as seen in this review, health technology interventions and decision support tools appear to be less implemented or less frequently reported. This does not mean that they are less developed than other knowledge products, but they are probably less commonly addressed in formal research or possibly less known by end users. These interventions, which are generally in the format of mobile-based or computerized-based interventions, are created to accelerate the accessibility and use of KT interventions. A decade ago, such interventions were considered new in the health domain, and it was reasonable that they were minimally implemented []. Currently, it is unclear why this situation persists, when health technology interventions and decision support tools are generally recognized as important aspects of care and the way of the future. The reasons may be attributed to the policy-making and funding level (health technologies are often short-term projects, ie, no long-term vision, nonexistence, and unpredictable changes in policies and regulations, financial constraints, eg, affordability, lack of infrastructure [such as office space, supplies, equipment, etc], human resource availability, and digital literacy) [,], or the implementation level (unawareness of the technology, perceived usefulness, ie, acceptability, etc) [,]. Finally, barriers may differ across settings and cultures [,].
The predominance of education-based, reminder, and audit and feedback implementation strategies suggests that they were prioritized based on existing barriers and facilitators of the implementation of the mentioned knowledge products. In fact, some of the most recent systematic and scoping reviews on the topic highlighted a lack of both provider awareness and knowledge of the existence of guidelines, and unfavorable attitudes about them [-], in response to which educational and audit and feedback strategies were judged to be suitable [,]. In contrast, a lack of access to guidelines and limited time available to providers was also mentioned [,], thereby calling for the use of decision support systems or reminders []. However, it is important to know whether these strategies are effective in implementing knowledge products. The included SRs demonstrated an all-directions effect, which was sometimes consistently positive or negative, or inconsistent, depending on factors such as single versus combined strategies [,,,] or type of comparator []. For example, in a study by Al Zoubi et al [], single educational strategies appeared to have a small effect, whereas multifaceted strategies that combine educational strategies and other types of strategies, such as reminders, appeared to be more effective, although inconsistent. Kovacs et al [] found the opposite result, showing that a single intervention is more effective. Others found that effectiveness may depend on the format in which education strategies are delivered; for example, by multimedia and computers [].
Adoption, which is also referred to as “uptake or utilization,” is “the intention, initial decision, or action to try or employ an innovation or evidence-based practice” []. This outcome occurs early or in the middle of the implementation process, is preceded by acceptability and appropriateness, and occurs at the same time as feasibility, followed by fidelity, implementation costs, penetration, and sustainability []. All proximal and distal implementation outcomes are important for measurement. The ongoing focus of the literature on the proximal outcome of adoption is more easily understood for recently introduced health technology interventions and strategies but is more difficult to explain when traditional strategies, such as education, are predominantly used.
Regarding the level of evidence of effectiveness, very few SRs have evaluated the level of evidence, as most reviews are narrative. The authors were unable to perform meta-analyses owing to high heterogeneity. In contrast, among the few reviews that assessed the level of evidence, most scored a low or very low grade. This makes it difficult to recognize potentially effective strategies and calls for more methodologically strong SRs to obtain reliable conclusions on the topic.
Strengths and Limitations
One strength of this SR is its broad objective, which included all EPOC strategies used to implement a variety of health knowledge products, with consideration given to the different implementation outcomes. No type of health care provider was excluded, and even if our target was primary health care, most included SRs covered both primary and secondary health care settings. We did not target any health area. We performed an extensive search and included both Cochrane and non-Cochrane reviews. It has been estimated that including only Cochrane reviews may lead to a loss or change in a median of 31% of the outcome data []. The objective of this phase was to characterize rather than measure effectiveness. Both types of SRs offered a large database of 81 reviews for future projects dealing with individual, unique RCTs. Therefore, we believe that our conclusions can be applied to many different contexts.
Few of the included reviews were of high or moderate methodological quality. The criteria lowering the scores may be linked to the unavailability of reporting guidelines at the time of publication or nonadherence to those guidelines when they were available. As per many other overviews, we used the AMSTAR tool and, as suggested, did so in a dual independent team format with a consensus process []. It is also possible to exclude reviews based on methodological quality issues when the aim is to produce a detailed picture of a topic [], as in our case.
With regard to the quality and completeness of the extracted information, in many of the included SRs, the categories of knowledge products, implementation strategies, and outcomes were not reported as per the standard taxonomies used, thereby requiring us to recategorize. This may have introduced some misclassification of information. We addressed this issue and its potential impact on our conclusion by piloting our data extraction process and reaching a consensus for all disagreements (by reviewing the discordant information together).
In the field of overviews, overlapping occurs when one primary study is included in more than one review or when more than one review addresses the same topic []. We cannot guarantee that our review will be free of overlapping issues. In addition, we did not evaluate the quality of evidence of outcomes for the included reviews, as we did not intend to demonstrate the effectiveness of the interventions. These 2 issues will be addressed in future projects on the effectiveness of strategies using the design of SRs of individual RCTs. For the comprehensiveness of our strategy, we searched 5 key databases in the field of intervention studies. In addition, we searched the reference lists of all included SRs. However, gray literature was not searched. For efficiency considerations, it was planned to update the search strategy in phase 3 of the project to avoid missing any recently published RCTs on the effectiveness of implementation strategies. We could look for gray literature in this phase.
Conclusions and Future Directions
Through this SR of SRs, we demonstrated that in the field of implementation, clinical practice guidelines and management, behavioral, or pharmacological health interventions are the most commonly implemented knowledge products, mainly through educational, reminder, and audit and feedback implementation strategies. However, the literature still focuses on the proximal outcomes of improving the adoption of knowledge products, generally with a limited level of evidence.
This SR aimed to provide insight into which knowledge products are frequently implemented, how they are implemented (implementation strategies), and the implementation outcomes measured, rather than providing information on the effectiveness of the implementation strategies. Therefore, in this step, we do not suggest changes in practice; rather, this review provides a good foundation for planning future research on effectiveness. Only detailed and contextualized information on knowledge products and implementation strategies will lead to changes in practice.
We constructed a database of SRs that may be used to strengthen the methodology for the SR of RCTs to overcome the issue of the variable effectiveness of commonly used implementation strategies, such as educational, reminder, and audit and feedback strategies. Future well-designed SRs of RCTs should fully describe the implementation strategy attributes of dose and intensity, format and duration of delivery, geographic location of interventions, and so on. In addition, qualitative studies and reviews involving a variety of collaborators from different domains and levels should be conducted to better understand the barriers and facilitators that contribute to why health technology interventions remain poorly implemented. Future implementation research should explore the entire cascade of implementation outcomes, including proximal and distal outcomes.
Acknowledgments
This project was supported by funds of the Canadian Institutes of Health Research, the Ministère de la Santé et des Services sociaux du Québec, and the Fonds de recherche du Québec Santé for the Québec Strategy for Patient-Oriented Research—Support for People and Patient-Oriented and Trials Unit. ABC was supported by Fonds de recherche du Québec Santé. FL holds a tier 1 Canada Research Chair in Shared Decision-Making and Knowledge Translation. Funders were not involved and had no role in developing the protocol; in the implementation of the project; or in the analysis, interpretation, reporting, and dissemination of results. We are grateful for Mr Théo Stefan for his support during screening of some citations and, for Mrs Marie-Joelle Cossi for assessing the methodological quality of some included reviews.
Authors' Contributions
HTVZ, JM, AB, AT, DK, CBU, NR, ABC, ED, LL, ST, SAG, and FL designed the protocol. NR designed the search strategy. FL, JM, LL, MD, ABC, and HTVZ reviewed the search strategy. NR implemented the search strategy. SAG, CBU, and ST screened the citations. SAG, CBU, JM, LP, and ST checked reviews for eligibility. HTVZ validated the list of included reviews. SAG, CBU, ED, ABC, MD, LP, and HTVZ assessed methodological quality of reviews. SAG, CBU, JM, ST, MD, LL, ED, and LK extracted data. LK and CBU analyzed data. CBU drafted the first version of manuscript. All authors critically reviewed and approved the manuscript.
Conflicts of Interest
None declared.
Multimedia Appendix 5
Knowledge products, implementation strategies, and outcomes measured in included reviews.
DOC File , 46 KB
Multimedia Appendix 6
A Measurement Tool to Assess Systematic Reviews 2 quality appraisal of included reviews.
DOC File , 83 KBReferences
- National Health Expenditure Data Historical. Centers for Medicare & Medicaid Services. 2020. URL: https://www.cms.gov/research-statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nationalhealthaccountshistorical [accessed 2021-05-19]
- Health spending. Canadian Institute for Health Information. 2021. URL: https://www.cihi.ca/en/topics/health-spending [accessed 2021-05-19]
- Glasziou P, Straus S, Brownlee S, Trevena L, Dans L, Guyatt G, et al. Evidence for underuse of effective medical services around the world. Lancet 2017 Jul 08;390(10090):169-177. [CrossRef] [Medline]
- Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, et al. Evidence for overuse of medical services around the world. Lancet 2017 Jul 08;390(10090):156-168 [FREE Full text] [CrossRef] [Medline]
- Rabin BA, Brownson RC. Terminology for dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford, UK: Oxford University Press; 2017.
- Rychetnik L, Hawe P, Waters E, Barratt A, Frommer M. A glossary for evidence based public health. J Epidemiol Community Health 2004 Jul;58(7):538-545 [FREE Full text] [CrossRef] [Medline]
- LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M. The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health 2012 Sep 07;12:751 [FREE Full text] [CrossRef] [Medline]
- EPOC Taxonomy. Cochrane – Effective Practice and Organisation of Care (EPOC). 2015. URL: https://epoc.cochrane.org/epoc-taxonomy [accessed 2020-06-29]
- Pantoja T, Opiyo N, Lewin S, Paulsen E, Ciapponi A, Wiysonge CS, et al. Implementation strategies for health systems in low-income countries: an overview of systematic reviews. Cochrane Database Syst Rev 2017 Sep 12;9(9):CD011086 [FREE Full text] [CrossRef] [Medline]
- Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013 Dec 01;8:139 [FREE Full text] [CrossRef] [Medline]
- Pfadenhauer LM, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A, Hofmann B, et al. Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement Sci 2017 Feb 15;12(1):21 [FREE Full text] [CrossRef] [Medline]
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009 Aug 07;4:50 [FREE Full text] [CrossRef] [Medline]
- Kirchner JE, Waltz TJ, Powell BJ, Smith JL, Proctor EK. Implementation strategies. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford, UK: Oxford University Press; 2017.
- Kaye AD, Okeagu CN, Pham AD, Silva RA, Hurley JJ, Arron BL, et al. Economic impact of COVID-19 pandemic on healthcare facilities and systems: international perspectives. Best Pract Res Clin Anaesthesiol 2021 Oct;35(3):293-306 [FREE Full text] [CrossRef] [Medline]
- Brouwers MC, Garcia K, Makarski J, Daraz L, Evidence Expert Panel, KT for Cancer Control in Canada Project Research Team. The landscape of knowledge translation interventions in cancer control: what do we know and where to next? A review of systematic reviews. Implement Sci 2011 Dec 20;6:130 [FREE Full text] [CrossRef] [Medline]
- Chan WV, Pearson TA, Bennett GC, Cushman WC, Gaziano TA, Gorman PN, et al. ACC/AHA special report: clinical practice guideline implementation strategies: a summary of systematic reviews by the NHLBI implementation science work group: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol 2017 Feb 28;69(8):1076-1092 [FREE Full text] [CrossRef] [Medline]
- Cheung A, Weir M, Mayhew A, Kozloff N, Brown K, Grimshaw J. Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior. Syst Rev 2012 Aug 16;1:36 [FREE Full text] [CrossRef] [Medline]
- Durieux P, Ravaud P, Dosquet P, Durocher A. [Effectiveness of clinical guideline implementation strategies: systematic review of systematic reviews]. Gastroenterol Clin Biol 2000 Nov;24(11):1018-1025 [FREE Full text] [Medline]
- Durieux P, Ravaud P. Efficacité des méthodes de mise en œuvre des recommandations médicales. Haute Autorité de Santé. 2014 Jul. URL: https://www.has-sante.fr/upload/docs/application/pdf/2014-09/efficacite_des_methodes_de_mise_en_oeuvre_des_recommandations_medicales.pdf [accessed 2019-10-30]
- Goldner EM, Jenkins EK, Fischer B. A narrative review of recent developments in knowledge translation and implications for mental health care providers. Can J Psychiatry 2014 Mar;59(3):160-169 [FREE Full text] [CrossRef] [Medline]
- Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care 2001 Aug;39(8 Suppl 2):II2-I45. [Medline]
- Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care--effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open 2015 Dec 23;5(12):e009993 [FREE Full text] [CrossRef] [Medline]
- Lewin S, Lavis JN, Oxman AD, Bastías G, Chopra M, Ciapponi A, et al. Supporting the delivery of cost-effective interventions in primary health-care systems in low-income and middle-income countries: an overview of systematic reviews. Lancet 2008 Sep 13;372(9642):928-939. [CrossRef] [Medline]
- Prior M, Guerin M, Grimmer-Somers K. The effectiveness of clinical guideline implementation strategies--a synthesis of systematic review findings. J Eval Clin Pract 2008 Oct;14(5):888-897. [CrossRef] [Medline]
- Pollock M, Fernandes RM, Becker LA, Featherstone R, Hartling L. What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Syst Rev 2016 Nov 14;5(1):190 [FREE Full text] [CrossRef] [Medline]
- Zomahoun HT, Massougbodji J, Bussières A, Thomas A, Kairy D, Uwizeye CB, et al. Improving the usefulness of evidence concerning the effectiveness of implementation strategies for knowledge products in primary healthcare: protocol for a series of systematic reviews. Syst Rev. 2020 May 19. URL: https://doi.org/10.17605/OSF.IO/EB8W2 [accessed 2021-03-26]
- Zomahoun HT, Massougbodji J, Bussières A, Thomas A, Kairy D, Uwizeye CB, et al. Improving the usefulness of evidence concerning the effectiveness of implementation strategies for knowledge products in primary healthcare: protocol for a series of systematic reviews. Syst Rev 2020 May 19;9(1):112 [FREE Full text] [CrossRef] [Medline]
- Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of reviews. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions. London, UK: Cochrane Collaboration and John Wiley & Sons; 2020.
- Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021 Mar 29;372:n160 [FREE Full text] [CrossRef] [Medline]
- McKenzie JE, Brennan SE, Ryan RE, Thomson HJ, Johnston RV, Thomas J. Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors. Cochrane handbook for systematic reviews of interventions. 2nd edition. Chichester, UK: John Wiley & Sons; 2019:33-66.
- About primary health care. Government of Canada. 2012 Aug 23. URL: https://www.canada.ca/en/health-canada/services/primary-health-care/about-primary-health-care.html [accessed 2020-09-28]
- Bhawra J, Skinner K. Examination of tools associated with the evaluation of knowledge uptake and utilization: a scoping review. Eval Program Plann 2020 Dec;83:101875 [FREE Full text] [CrossRef] [Medline]
- Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof 2006;26(1):13-24. [CrossRef] [Medline]
- Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar;38(2):65-76 [FREE Full text] [CrossRef] [Medline]
- Lewis CC, Proctor EK, Brownson RC. Measurement issues in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford, UK: Oxford University Press; 2017.
- McKibbon KA. Evidence-based practice. Bull Med Libr Assoc 1998 Jul;86(3):396-401 [FREE Full text] [Medline]
- Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Institute of Medicine. In: Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editors. Clinical Practice Guidelines We Can Trust. Washington, DC, USA: National Academies Press (US); 2011.
- Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: an evolving concept. Am J Prev Med 2004 Dec;27(5):417-421. [CrossRef] [Medline]
- Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017 Sep 21;358:j4008 [FREE Full text] [CrossRef] [Medline]
- Abdullah G, Rossy D, Ploeg J, Davies B, Higuchi K, Sikora L, et al. Measuring the effectiveness of mentoring as a knowledge translation intervention for implementing empirical evidence: a systematic review. Worldviews Evid Based Nurs 2014 Oct;11(5):284-300 [FREE Full text] [CrossRef] [Medline]
- Al Zoubi FM, Menon A, Mayo N, Bussières AE. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res 2018 Jun 08;18(1):435 [FREE Full text] [CrossRef] [Medline]
- Albrecht L, Archibald M, Snelgrove-Clarke E, Scott SD. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs 2016;31(3):235-254. [CrossRef] [Medline]
- Bacci JL, Bigham KA, Dillon‐Sumner L, Ferreri S, Frail CK, Hamada CY, et al. Community pharmacist patient care services: a systematic review of approaches used for implementation and evaluation. J Am Coll Clin Pharm 2019 Jun 17;2(4):423-432 [FREE Full text] [CrossRef]
- Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev 2015 Apr 29;2015(4):CD005470 [FREE Full text] [CrossRef] [Medline]
- Barwick MA, Schachter HM, Bennett LM, McGowan J, Ly M, Wilson A, et al. Knowledge translation efforts in child and youth mental health: a systematic review. J Evid Based Soc Work 2012;9(4):369-395 [FREE Full text] [CrossRef] [Medline]
- Boersma P, van Weert JC, Lakerveld J, Dröes RM. The art of successful implementation of psychosocial interventions in residential dementia care: a systematic review of the literature based on the RE-AIM framework - CORRIGENDUM. Int Psychogeriatr 2015 Jan;27(1):37. [CrossRef] [Medline]
- Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012 Jul 03;157(1):29-43 [FREE Full text] [CrossRef] [Medline]
- Brusamento S, Legido-Quigley H, Panteli D, Turk E, Knai C, Saliba V, et al. Assessing the effectiveness of strategies to implement clinical guidelines for the management of chronic diseases at primary care level in EU Member States: a systematic review. Health Policy 2012 Oct;107(2-3):168-183. [CrossRef] [Medline]
- Bywood P, Lunnay B, Roche A. Effectiveness of opinion leaders for getting research into practice in the alcohol and other drugs field: results from a systematic literature review. Drugs (Abingdon Engl) 2009 Jul 10;16(3):205-216 [FREE Full text] [CrossRef]
- Campbell A, Louie-Poon S, Slater L, Scott SD. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs 2019;47:114-120. [CrossRef] [Medline]
- Chaillet N, Dubé E, Dugas M, Audibert F, Tourigny C, Fraser WD, et al. Evidence-based strategies for implementing guidelines in obstetrics: a systematic review. Obstet Gynecol 2006 Nov;108(5):1234-1245. [CrossRef] [Medline]
- Ciliska D, Robinson P, Armour T, Ellis P, Brouwers M, Gauld M, et al. Diffusion and dissemination of evidence-based dietary strategies for the prevention of cancer. Nutr J 2005 Apr 08;4:13 [FREE Full text] [CrossRef] [Medline]
- Colquhoun HL, Lamontagne ME, Duncan EA, Fiander M, Champagne C, Grimshaw JM. A systematic review of interventions to increase the use of standardized outcome measures by rehabilitation professionals. Clin Rehabil 2017 Mar;31(3):299-309. [CrossRef] [Medline]
- Davis DA, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ 1997 Aug 15;157(4):408-416 [FREE Full text] [Medline]
- De Angelis G, Davies B, King J, McEwan J, Cavallo S, Loew L, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ 2016 Nov 30;2(2):e16 [FREE Full text] [CrossRef] [Medline]
- Dexheimer JW, Borycki EM, Chiu KW, Johnson KB, Aronsky D. A systematic review of the implementation and impact of asthma protocols. BMC Med Inform Decis Mak 2014 Sep 09;14:82 [FREE Full text] [CrossRef] [Medline]
- Dexheimer JW, Talbot TR, Sanders DL, Rosenbloom ST, Aronsky D. Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. J Am Med Inform Assoc 2008;15(3):311-320 [FREE Full text] [CrossRef] [Medline]
- Dwamena F, Holmes-Rovner M, Gaulden CM, Jorgenson S, Sadigh G, Sikorskii A, et al. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochrane Database Syst Rev 2012 Dec 12;12:CD003267. [CrossRef] [Medline]
- Espallargues M, Valderas JM, Alonso J. Provision of feedback on perceived health status to health care professionals: a systematic review of its impact. Med Care 2000 Feb;38(2):175-186. [CrossRef] [Medline]
- Flodgren G, Conterno LO, Mayhew A, Omar O, Pereira CR, Shepperd S. Interventions to improve professional adherence to guidelines for prevention of device-related infections. Cochrane Database Syst Rev 2013 Mar 28(3):CD006559. [CrossRef] [Medline]
- Flodgren G, Gonçalves-Bradley DC, Summerbell CD. Interventions to change the behaviour of health professionals and the organisation of care to promote weight reduction in children and adults with overweight or obesity. Cochrane Database Syst Rev 2017 Nov 30;11(11):CD000984 [FREE Full text] [CrossRef] [Medline]
- Flodgren G, Hall AM, Goulding L, Eccles MP, Grimshaw JM, Leng GC, et al. Tools developed and disseminated by guideline producers to promote the uptake of their guidelines. Cochrane Database Syst Rev 2016 Aug 22(8):CD010669. [CrossRef] [Medline]
- Flodgren G, O'Brien MA, Parmelli E, Grimshaw JM. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2019 Jun 24;6(6):CD000125 [FREE Full text] [CrossRef] [Medline]
- Forman-Hoffman VL, Middleton JC, McKeeman JL, Stambaugh LF, Christian RB, Gaynes BN, et al. Quality improvement, implementation, and dissemination strategies to improve mental health care for children and adolescents: a systematic review. Implement Sci 2017 Jul 24;12(1):93 [FREE Full text] [CrossRef] [Medline]
- Gagnon MP, Légaré F, Labrecque M, Frémont P, Pluye P, Gagnon J, et al. Interventions for promoting information and communication technologies adoption in healthcare professionals. Cochrane Database Syst Rev 2009 Jan 21(1):CD006093 [FREE Full text] [CrossRef] [Medline]
- Gifford W, Davies B, Edwards N, Griffin P, Lybanon V. Managerial leadership for nurses' use of research evidence: an integrative review of the literature. Worldviews Evid Based Nurs 2007;4(3):126-145. [CrossRef] [Medline]
- Gould DJ, Moralejo D, Drey N, Chudleigh JH, Taljaard M. Interventions to improve hand hygiene compliance in patient care. Cochrane Database Syst Rev 2017 Sep 01;9(9):CD005186 [FREE Full text] [CrossRef] [Medline]
- Goveia J, Van Stiphout F, Cheung Z, Kamta B, Keijsers C, Valk G, et al. Educational interventions to improve the meaningful use of Electronic Health Records: a review of the literature: BEME Guide No. 29. Med Teach 2013 Nov;35(11):e1551-e1560. [CrossRef] [Medline]
- Gross PA, Pujat D. Implementing practice guidelines for appropriate antimicrobial usage: a systematic review. Med Care 2001 Aug;39(8 Suppl 2):II55-II69. [CrossRef] [Medline]
- Häggman-Laitila A, Mattila LR, Melender HL. A systematic review of the outcomes of educational interventions relevant to nurses with simultaneous strategies for guideline implementation. J Clin Nurs 2017 Feb;26(3-4):320-340. [CrossRef] [Medline]
- Hamade N, Terry A, Malvankar-Mehta M. Interventions to improve the use of EMRs in primary health care: a systematic review and meta-analysis. BMJ Health Care Inform 2019 May;26(1):e000023 [FREE Full text] [CrossRef] [Medline]
- Heselmans A, Van de Velde S, Donceel P, Aertgeerts B, Ramaekers D. Effectiveness of electronic guideline-based implementation systems in ambulatory care settings - a systematic review. Implement Sci 2009 Dec 30;4:82 [FREE Full text] [CrossRef] [Medline]
- Hoomans T, Evers SM, Ament AJ, Hübben MA, van der Weijden T, Grimshaw JM, et al. The methodological quality of economic evaluations of guideline implementation into clinical practice: a systematic review of empiric studies. Value Health 2007;10(4):305-316 [FREE Full text] [CrossRef] [Medline]
- Imamura M, Kanguru L, Penfold S, Stokes T, Camosso-Stefinovic J, Shaw B, et al. A systematic review of implementation strategies to deliver guidelines on obstetric care practice in low- and middle-income countries. Int J Gynaecol Obstet 2017 Jan;136(1):19-28. [CrossRef] [Medline]
- Ince P, Haddock G, Tai S. A systematic review of the implementation of recommended psychological interventions for schizophrenia: rates, barriers, and improvement strategies. Psychol Psychother 2016 Sep;89(3):324-350. [CrossRef] [Medline]
- Ista E, van Dijk M, van Achterberg T. Do implementation strategies increase adherence to pain assessment in hospitals? A systematic review. Int J Nurs Stud 2013 Apr;50(4):552-568. [CrossRef] [Medline]
- Jeffery RA, To MJ, Hayduk-Costa G, Cameron A, Taylor C, Van Zoost C, et al. Interventions to improve adherence to cardiovascular disease guidelines: a systematic review. BMC Fam Pract 2015 Oct 22;16:147. [CrossRef] [Medline]
- Jensen CE, Jensen MB, Riis A, Petersen KD. Systematic review of the cost-effectiveness of implementing guidelines on low back pain management in primary care: is transferability to other countries possible? BMJ Open 2016 Jun 07;6(6):e011042 [FREE Full text] [CrossRef] [Medline]
- Jones CA, Roop SC, Pohar SL, Albrecht L, Scott SD. Translating knowledge in rehabilitation: systematic review. Phys Ther 2015 Apr;95(4):663-677 [FREE Full text] [CrossRef] [Medline]
- Jones L, Bates G, McCoy E, Beynon C, McVeigh J, Bellis MA. Effectiveness of interventions to increase hepatitis C testing uptake among high-risk groups: a systematic review. Eur J Public Health 2014 Oct;24(5):781-788. [CrossRef] [Medline]
- Kovacs E, Strobl R, Phillips A, Stephan AJ, Müller M, Gensichen J, et al. Systematic review and meta-analysis of the effectiveness of implementation strategies for non-communicable disease guidelines in primary health care. J Gen Intern Med 2018 Jul;33(7):1142-1154 [FREE Full text] [CrossRef] [Medline]
- Légaré F, Turcotte S, Stacey D, Ratté S, Kryworuchko J, Graham ID. Patients' perceptions of sharing in decisions: a systematic review of interventions to enhance shared decision making in routine clinical practice. Patient 2012;5(1):1-19. [CrossRef] [Medline]
- Lineker SC, Husted JA. Educational interventions for implementation of arthritis clinical practice guidelines in primary care: effects on health professional behavior. J Rheumatol 2010 Aug 01;37(8):1562-1569. [CrossRef] [Medline]
- Luangasanatip N, Hongsuwan M, Limmathurotsakul D, Lubell Y, Lee AS, Harbarth S, et al. Comparative efficacy of interventions to promote hand hygiene in hospital: systematic review and network meta-analysis. BMJ 2015 Jul 28;351:h3728 [FREE Full text] [CrossRef] [Medline]
- Medves J, Godfrey C, Turner C, Paterson M, Harrison M, MacKenzie L, et al. Systematic review of practice guideline dissemination and implementation strategies for healthcare teams and team-based practice. Int J Evid Based Healthc 2010 Jun;8(2):79-89. [CrossRef] [Medline]
- Menon A, Korner-Bitensky N, Kastner M, McKibbon KA, Straus S. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med 2009 Nov;41(13):1024-1032 [FREE Full text] [CrossRef] [Medline]
- Murthy L, Shepperd S, Clarke MJ, Garner SE, Lavis JN, Perrier L, et al. Interventions to improve the use of systematic reviews in decision-making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 2012 Sep 12(9):CD009401. [CrossRef] [Medline]
- Nilsen P, Aalto M, Bendtsen P, Seppä K. Effectiveness of strategies to implement brief alcohol intervention in primary healthcare. A systematic review. Scand J Prim Health Care 2006 Mar;24(1):5-15. [CrossRef] [Medline]
- Noonan VK, Wolfe DL, Thorogood NP, Park SE, Hsieh JT, Eng JJ, SCIRE Research Team. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord 2014 Aug;52(8):578-587 [FREE Full text] [CrossRef] [Medline]
- Novins DK, Green AE, Legha RK, Aarons GA. Dissemination and implementation of evidence-based practices for child and adolescent mental health: a systematic review. J Am Acad Child Adolesc Psychiatry 2013 Oct;52(10):1009-25.e18 [FREE Full text] [CrossRef] [Medline]
- Okelo SO, Butz AM, Sharma R, Diette GB, Pitts SI, King TM, et al. Interventions to modify health care provider adherence to asthma guidelines: a systematic review. Pediatrics 2013 Sep;132(3):517-534 [FREE Full text] [CrossRef] [Medline]
- Ospina MB, Taenzer P, Rashiq S, MacDermid JC, Carr E, Chojecki D, et al. A systematic review of the effectiveness of knowledge translation interventions for chronic noncancer pain management. Pain Res Manag 2013;18(6):e129-e141 [FREE Full text] [CrossRef] [Medline]
- Pearson SA, Moxey A, Robertson J, Hains I, Williamson M, Reeve J, et al. Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990-2007). BMC Health Serv Res 2009 Aug 28;9:154 [FREE Full text] [CrossRef] [Medline]
- Perrier L, Mrklas K, Shepperd S, Dobbins M, McKibbon KA, Straus SE. Interventions encouraging the use of systematic reviews in clinical decision-making: a systematic review. J Gen Intern Med 2011 Apr;26(4):419-426 [FREE Full text] [CrossRef] [Medline]
- Perry M, Drašković I, Lucassen P, Vernooij-Dassen M, van Achterberg T, Rikkert MO. Effects of educational interventions on primary dementia care: a systematic review. Int J Geriatr Psychiatry 2011 Jan;26(1):1-11. [CrossRef] [Medline]
- Pham C, Lizarondo L, Karnon J, Aromataris E, Munn Z, Gibb C, et al. Strategies for implementing shared decision making in elective surgery by health care practitioners: a systematic review. J Eval Clin Pract 2020 Apr;26(2):582-601. [CrossRef] [Medline]
- Powell BJ, Proctor EK, Glass JE. A systematic review of strategies for implementing empirically supported mental health interventions. Res Soc Work Pract 2014 Mar 01;24(2):192-212 [FREE Full text] [CrossRef] [Medline]
- Rosen CS, Matthieu MM, Wiltsey Stirman S, Cook JM, Landes S, Bernardy NC, et al. A review of studies on the system-wide implementation of evidence-based psychotherapies for posttraumatic stress disorder in the veterans health administration. Adm Policy Ment Health 2016 Nov;43(6):957-977. [CrossRef] [Medline]
- Scott SD, Albrecht L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci 2012 Jul 25;7:70 [FREE Full text] [CrossRef] [Medline]
- Shanbhag D, Graham ID, Harlos K, Haynes RB, Gabizon I, Connolly SJ, et al. Effectiveness of implementation interventions in improving physician adherence to guideline recommendations in heart failure: a systematic review. BMJ Open 2018 Mar 06;8(3):e017765 [FREE Full text] [CrossRef] [Medline]
- Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc 1999;6(2):104-114 [FREE Full text] [CrossRef] [Medline]
- Siddiqui MR, Sajid MS, Khatri K, Kanri B, Cheek E, Baig MK. The role of physician reminders in faecal occult blood testing for colorectal cancer screening. Eur J Gen Pract 2011 Dec;17(4):221-228. [CrossRef] [Medline]
- Smeets HM, Hoes AH, de Wit NJ. Effectiveness and costs of implementation strategies to reduce acid suppressive drug prescriptions: a systematic review. BMC Health Serv Res 2007 Nov 05;7:177 [FREE Full text] [CrossRef] [Medline]
- Smolders M, Laurant M, Roberge P, van Balkom A, van Rijswijk E, Bower P, et al. Knowledge transfer and improvement of primary and ambulatory care for patients with anxiety. Can J Psychiatry 2008 May;53(5):277-293. [CrossRef] [Medline]
- Soumerai SB, McLaughlin TJ, Avorn J. Improving drug prescribing in primary care: a critical analysis of the experimental literature. Milbank Q 1989;67(2):268-317. [Medline]
- Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci 2011 Aug 03;6:87 [FREE Full text] [CrossRef] [Medline]
- Sunderji N, Ion A, Huynh D, Benassi P, Ghavam-Rassoul A, Carvalhal A. Advancing integrated care through psychiatric workforce development: a systematic review of educational interventions to train psychiatrists in integrated care. Can J Psychiatry 2018 Aug;63(8):513-525 [FREE Full text] [CrossRef] [Medline]
- Thomas L, Cullum N, McColl E, Rousseau N, Soutter J, Steen N. Guidelines in professions allied to medicine. Cochrane Database Syst Rev 2000(2):CD000349. [CrossRef] [Medline]
- Thompson DS, Estabrooks CA, Scott-Findlay S, Moore K, Wallin L. Interventions aimed at increasing research use in nursing: a systematic review. Implement Sci 2007 May 11;2:15 [FREE Full text] [CrossRef] [Medline]
- Tudor Car L, Soong A, Kyaw BM, Chua KL, Low-Beer N, Majeed A. Health professions digital education on clinical practice guidelines: a systematic review by Digital Health Education collaboration. BMC Med 2019 Jul 18;17(1):139 [FREE Full text] [CrossRef] [Medline]
- Unverzagt S, Oemler M, Braun K, Klement A. Strategies for guideline implementation in primary care focusing on patients with cardiovascular disease: a systematic review. Fam Pract 2014 Jun;31(3):247-266. [CrossRef] [Medline]
- van der Wees PJ, Jamtvedt G, Rebbeck T, de Bie RA, Dekker J, Hendriks EJ. Multifaceted strategies may increase implementation of physiotherapy clinical guidelines: a systematic review. Aust J Physiother 2008;54(4):233-241. [CrossRef] [Medline]
- van Steenkiste B, Grol R, van der Weijden T. Systematic review of implementation strategies for risk tables in the prevention of cardiovascular diseases. Vasc Health Risk Manag 2008;4(3):535-545 [FREE Full text] [CrossRef] [Medline]
- Watkins K, Wood H, Schneider CR, Clifford R. Effectiveness of implementation strategies for clinical guidelines to community pharmacy: a systematic review. Implement Sci 2015 Oct 29;10:151 [FREE Full text] [CrossRef] [Medline]
- Weinmann S, Koesters M, Becker T. Effects of implementation of psychiatric guidelines on provider performance and patient outcome: systematic review. Acta Psychiatr Scand 2007 Jun;115(6):420-433. [CrossRef] [Medline]
- Wensing M, van der Weijden T, Grol R. Implementing guidelines and innovations in general practice: which interventions are effective? Br J Gen Pract 1998 Feb;48(427):991-997 [FREE Full text] [Medline]
- Wilbur K, Elmubark A, Shabana S. Systematic review of standardized patient use in continuing medical education. J Contin Educ Health Prof 2018;38(1):3-10. [CrossRef] [Medline]
- Wilson CL, Johnson D, Oakley E, Paediatric Research in Emergency Departments International Collaborative (PREDICT) network. Knowledge translation studies in paediatric emergency medicine: a systematic review of the literature. J Paediatr Child Health 2016 Feb;52(2):112-125. [CrossRef] [Medline]
- Wuchner SS. Integrative review of implementation strategies for translation of research-based evidence by nurses. Clin Nurse Spec 2014;28(4):214-223. [CrossRef] [Medline]
- Zaher E, Ratnapalan S. Practice-based small group learning programs: systematic review. Can Fam Physician 2012 Jun;58(6):637-e316 [FREE Full text] [Medline]
- Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan 2016 Nov;31(9):1310-1325. [CrossRef] [Medline]
- Early J, Gonzalez C, Gordon-Dseagu V, Robles-Calderon L. Use of mobile health (mHealth) technologies and interventions among community health workers globally: a scoping review. Health Promot Pract 2019 Nov;20(6):805-817. [CrossRef] [Medline]
- Almazrou SH, Alfaifi SI, Alfaifi SH, Hakami LE, Al-Aqeel SA. Barriers to and facilitators of adherence to clinical practice guidelines in the Middle East and North Africa region: a systematic review. Healthcare (Basel) 2020 Dec 15;8(4):564 [FREE Full text] [CrossRef] [Medline]
- Correa VC, Lugo-Agudelo LH, Aguirre-Acevedo DC, Contreras JA, Borrero AM, Patiño-Lugo DF, et al. Individual, health system, and contextual barriers and facilitators for the implementation of clinical practice guidelines: a systematic metareview. Health Res Policy Syst 2020 Jun 29;18(1):74 [FREE Full text] [CrossRef] [Medline]
- Fischer F, Lange K, Klose K, Greiner W, Kraemer A. Barriers and strategies in guideline implementation-a scoping review. Healthcare (Basel) 2016 Jun 29;4(3):36 [FREE Full text] [CrossRef] [Medline]
- Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions. Syst Rev 2019 Jan 11;8(1):18 [FREE Full text] [CrossRef] [Medline]
- Pollock M, Fernandes RM, Hartling L. Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions. BMC Med Res Methodol 2017 Mar 23;17(1):48 [FREE Full text] [CrossRef] [Medline]
- Biondi-Zoccai G. Umbrella Reviews: Evidence Synthesis with Overviews of Reviews and Meta-Epidemiologic Studies. Berlin, Germany: Springer; Feb 3, 2016.
Abbreviations
| AMSTAR: A Measurement Tool to Assess Systematic Reviews |
| EPOC: Effective Practice and Organization of Care |
| KT: knowledge translation |
| PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
| RCT: randomized controlled trial |
| SR: systematic review |
Edited by T Leung, G Eysenbach; submitted 02.04.22; peer-reviewed by D Tran, A Karabukayeva; comments to author 28.04.22; revised version received 20.05.22; accepted 30.05.22; published 11.07.22
Copyright©Claude Bernard Uwizeye, Hervé Tchala Vignon Zomahoun, André Bussières, Aliki Thomas, Dahlia Kairy, José Massougbodji, Nathalie Rheault, Sébastien Tchoubi, Leonel Philibert, Serigne Abib Gaye, Lobna Khadraoui, Ali Ben Charif, Ella Diendéré, Léa Langlois, Michèle Dugas, France Légaré. Originally published in the Interactive Journal of Medical Research (https://www.i-jmr.org/), 11.07.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.i-jmr.org/, as well as this copyright and license information must be included.