Original Research

Searching for published protocols of randomized controlled trials in primary health care: an empirical systematic approach

AUTHORS

Evridiki Papagiannopoulou1 RN, PhD Candidate

Christina Antoniadou2 MD, Research Associate

Eleni Ntalaouti3 MD, Research Associate

Evangelia Ntzani4 MD, Associate Professor

Konstantinos Siamopoulos5 MD, Professor Emeritus

name here
Athina Tatsioni6
PhD, Associate Professor *

CORRESPONDENCE

*A/Prof Athina Tatsioni

AFFILIATIONS

1, 2, 3, 6 Research Unit for General Medicine and Primary Health Care, Faculty of Medicine, School of Health Sciences, University of Ioannina, Ioannina, Greece

4 Department of Hygiene and Epidemiology, Faculty of Medicine, School of Health Sciences, University of Ioannina, Ioannina, Greece; and Center for Evidence-Based Medicine, Department of Health Services, Policy and Practice, School of Public Health, Brown University, Providence, RI, USA

5 Faculty of Medicine, School of Health Sciences, University of Ioannina, Ioannina, Greece

PUBLISHED

25 February 2020 Volume 20 Issue 1

HISTORY

RECEIVED: 22 September 2018

REVISED: 15 September 2019

ACCEPTED: 25 November 2019

CITATION

Papagiannopoulou E, Antoniadou C, Ntalaouti E, Ntzani E, Siamopoulos K, Tatsioni A.  Searching for published protocols of randomized controlled trials in primary health care: an empirical systematic approach. Rural and Remote Health 2020; 20: 5108. https://doi.org/10.22605/RRH5108

AUTHOR CONTRIBUTIONSgo to url

This work is licensed under a Creative Commons Attribution 4.0 International Licence


abstract:

Introduction:  Electronic search filters on family medicine or general practice studies have been developed and validated in previous work. However, there has been no systematic effort to specifically identify and record protocols of randomized controlled trials (RCT) protocols in primary health care (PHC). The aim of the present study was to systematically identify published RCT protocols in PHC and capture information about specific protocol characteristics that may describe this field.
Methods:  PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Scopus from inception to December 2014 were systematically searched. Protocols of RCTs that were published in English and were relevant to PHC were considered as eligible. Protocols referred either to a mixed population, or to an intervention including a specialized part as well as pilot or feasibility trial protocols, were excluded. Specific protocol characteristics including publication year, country, prospective registration, funding, and publication sources were extracted.
Results:  The final database included 628 published RCT protocols (median publication year 2011; interquartile range 2009–2013). The majority of protocols were designed in the UK (n=141, 22.5%), the Netherlands (n=105, 16.7%), and USA (n=93, 14.8%). Research was mainly funded by the government (n=408, 65.0%) while 45 protocols (7.2%) included industry as the funding source. Two registries – International Standard Randomised Controlled Trials Number Registry (245 (42.9%)) and ClinicalTrials.gov (209 (36.6%)) – indexed most of the protocols. Journals from several scientific fields published the articles; the field of ‘Primary Health Care Medicine, General and Internal’ included 69 (11.0%) articles.
Conclusion:  A compilation of published RCT protocols on PHC was feasible. The majority of protocols on PHC were published over the past 10 years, funded by the government and designed in three main countries.

Keywords:

clinical randomized controlled trial, implementation research, primary health care, protocols.

full article:

Introduction

Prospective reporting of clinical trial protocols – either in a trial registry or as a published article – has been encouraged in recent decades. Several proponents suggested that prospective reporting of trial design might enhance transparency and therefore improve accountability in the conduct and reporting of research1. In addition, researchers can have access to the design of ongoing trials that may be relevant to their work. Furthermore, the general public and other stakeholders can be informed of ongoing trials that may influence their decisions and policies in the future2-4.

Robust conclusions from randomized controlled trials (RCTs) are pivotal for primary health care (PHC) to support both clinical decision making and healthcare policy5. RCTs in PHC are not exempted from prospective reporting; their protocols may be indexed in clinical trial registries or as published articles before reporting the results6. However, the search for protocols of RCTs in PHC may be challenging and time consuming because there is no globally accepted algorithm on identifying studies with population and intervention relevant to PHC.

The characteristic position of PHC has been well noted7,8. Previous studies have described essential spectrum differences between PHC and specialized, referred (secondary) care9. However, the diversity of clinical areas and infrastructure among different countries in PHC complicates the identification of protocols designed for this field6. Previous articles have developed and validated electronic search filters on family medicine or general practice studies10-13. However, to the authors’ knowledge, there has been no systematic effort to specifically identify and record protocols of RCTs in this field. Systematic searches that may successfully identify protocols on PHC may shed light on the direction of current research in PHC, reveal potential gaps, structure PHC research agenda and guide future developing of research capacity.

The present study aimed at systematically identifying published RCT protocols in PHC. In addition, information about specific characteristics including publication year, country, prospective registration, funding, and publication sources was captured. This compilation of PHC trial protocols is part of an ongoing project on assessing pragmatic/explanatory continuum in the design of published RCT protocols on PHC.

Methods

Search strategy

Electronic databases were searched including PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Scopus from inception (1966 for PubMed, 1981 for CINAHL, and 1996 for Scopus) to December 2014 (last search January 2015). A search strategy was created for published RCT protocols on PHC including keywords related to family medicine/general practice and PHC combined with the Cochrane Collaboration search strategy for published RCTs. The final strategy is shown in Appendix I. A similar systematic search was conducted on CINAHL after excluding articles registered in MEDLINE. The final strategy is shown in Appendix I. Finally, Scopus database was used to enhance search results. The exact strategy is shown in Appendix I. In a Scopus search, after excluding articles registered in MEDLINE, the journals for the remaining articles checked; the articles were excluded if they had been published in journals indexed in PubMed and therefore were likely to have already been screened among the PubMed publications. All articles that were included in journals not indexed in PubMed were screened.

Three investigators (EP, EEN, and AT) independently performed a pilot screening based on the title and/or abstract for the first 500 items in PubMed to identify potential disagreements in the screening process. After settling potential discrepancies with consensus, one investigator (EP) completed the screening of all titles and/or abstracts in PubMed, CINAHL, and Scopus. A second investigator (EEN or AT) checked on the items that the first investigator could not decide upon. For items for which both investigators were not able to decide on their eligibility, the authors retrieved the full text of the article. Three investigators (EP, AC, and EN) independently screened articles in full text. A fourth investigator (AT) resolved any discrepancies. For those articles that investigators were not able to decide on their eligibility based on full text, the authors electronically contacted the article’s corresponding author to request additional information.

Inclusion criteria

Published protocols of RCTs that were relevant to PHC were considered as eligible. An article that reported the methodology of an RCT without including any part of the results was considered as a protocol. An article was considered relevant to PHC if the investigators clearly stated that they would recruit participants from PHC, general practice, or family medicine, and that PHC professional general practitioners or family physicians would be involved in the intervention. In case investigators referred either to a mixed population (PHC and secondary/tertiary care) or an intervention including a specialized part, the article was excluded. There were not any exclusion criteria about the type of professionals involved; the investigators just had to clearly mention that they worked in PHC. Protocols on pilot or feasibility trials as well as protocols that described non-randomized or pseudo-randomized studies were excluded. Only publications written in English were included. No limitations on publication time were set.

Data extraction

For each eligible article, the extracted items included PMID, first author’s name, publication year, and the country in which it would be conducted; whether the protocol was registered and, if it was, the database it was indexed in; the registration date, the registration number, the funding source using the exact wording in the publication; and the journal in which it was published and its field according to Web of Science. In addition, each country was categorized according to the world region. The following regions were used: Western Europe, North America, Mediterranean, Oceania, Central Europe, Scandinavia, Africa, East Asia, South Asia, Central and South America, and the Middle East. Based on the funding source, the studies were categorized in the following groups: government or public funding, funded by non-governmental organization or institute, mixed funding excluding industry, mixed funding including industry, and funded by industry only. Additional categories included the studies that clearly stated that they were not funded and the publications that did not mention funding at all. Finally, the health topic covered by each protocol was recorded.  

Two independent investigators (CA and EN) extracted the data. Discrepancies were resolved by consensus; where a consensus could not be reached, a third arbitrator (AT) participated in the final decision.

Statistical analysis

Data were presented as absolute numbers and frequencies for binary and categorical variables, and as medians with interquartile range (IQR) for continuous variables. All analyses were performed with the Statistical Package for the Social Sciences v21.0 (IBM; http://www.spss.com.au).

Ethics approval

The study did not involve human subjects, and thus it was exempt from documentation of informed consent and institutional review board approval. 

Results

Eligible articles

The search yielded a total of 26 571 items in PubMed, 268 items in CINAHL journals that were not indexed in MEDLINE, and 670 items in Scopus that were published in 24 journals not indexed in MEDLINE or PubMed. Based on the title and/or abstract, 25 739 articles retrieved in PubMed and all the items found in CINAHL or Scopus were excluded as not relevant, mainly because they were not protocols. Notably, there was no study protocol among the articles registered only in CINAHL or Scopus. Thus, 832 articles were evaluated in full text. Out of the 832 publications that were potentially eligible, 81 were excluded because they were not relevant to PHC; 61 articles because they referred to mixed population and/or intervention including secondary health care; 27 articles as pilot/feasibility studies; 14 because they were not RCTs; and 1 publication because it was not a protocol.

For 29 articles, their eligibility for PHC after reading the full text was unclear and therefore the corresponding author was contacted by e-mail. A reply was received from 19 authors; 10 out of 19 responded that their article was not relevant to PHC.

Thus, 184 out of the 832 potentially eligible publications were excluded. In addition, 10 out of the 29 articles for which a response from their corresponding authors was not received were excluded; and one additional article that could not be retrieved in full text.

Out of the 27 509 potentially relevant items in PubMed, CINAHL, and Scopus, the final database included 628 eligible published RCT protocols on PHC after excluding 26,677 items based on title and /or abstract as not relevant; and after excluding 203 articles based on full text screening, and one article, which could not be retrieved in full text (Fig1).

table image Figure 1:  Flow chart for eligible articles.

Characteristics of eligible published protocols

For the 628 eligible articles, median publication year was 2011 (IQR 2009–2013). Out of the 628 published protocols, 572 (91.1%), reported that they were prospectively registered in a database of clinical studies (Table 1). Out of the 628 protocols, 262 (41.7%) were designed in Western Europe, 117 (18.6%) in North America, 77 (12.3%) in the Mediterranean, 62 (9.9%) in Oceania, 42 (6.7%) in Central Europe, 33 (5.3%) in Scandinavia, and 10 (1.6%) in Africa. The authors found five protocols (0.8%) conducted in eastern Asia including China and Japan, 4 (0.6%) in South-East Asia, 4 (0.6%) in South Asia including India, 2 (0.3%) in Central and South America, and 2 (0.3%) in the Middle East. In addition, 8 (1.3%) protocols were conducted in several regions (Table 1). Out of the 628 articles, 408 (65.0%) articles reported that they were funded by the government; 76 studies (12.1%) were supported by non-governmental organizations or institutes; 64 (10.2%) studies reported funding both by the government and by non-governmental organizations or institutes; while 30 (4.8%) articles reported funding by several sponsors including industry; and 15 (2.4%) that they were exclusively funded by industry. In one protocol, authors clearly stated that there was no funding. In 35 (5.6%) articles, authors did not mention any information about the funding source (Table 1). When the funding sources per geographical region were checked, no differences from the overall pattern were observed (Supplementary table S1).

Out of the 571 published protocols that mentioned registration in a clinical study database, 245 (42.9%) were indexed in the International Standard Randomised Controlled Trials Number (ISRCTN) Registry, 209 (36.6%) in ClinicalTrials.gov, 53 (9.3%) in the Australian New Zealand Clinical Trials Registry, and 47 (8.2%) in the Netherlands National Trial Registry (Table 2). The 10 countries that published the highest number of protocols are presented in Table 3. Specifically, 141 (22.5%) articles were from the UK, 105 (16.7%) from the Netherlands, 93 (14.8%) from the USA and 69 (11.0%) from Spain (Table 3; all countries are represented in Supplementary table S2). Australia, Germany, Canada, Denmark, and Norway were also among the top 10 countries. Nine (1.4%) articles included protocols that were conducted in several countries.

Table 4 presents the 10 field categories according to Web of Science, which included most of the journals that published the 628 protocols. Specifically, 117 (18.6%) articles were published in journals included in the ‘Medicine, Research and Experimental’ field, 105 (16.7%) in ‘Health Care Sciences and Services’, 102 (16.2%) in ‘Public, Environmental and Occupational Health’, 69 (11.0%) in ‘Primary Health Care’, 38 (6.1%) in ‘Orthopedics Rheumatology’, 27 (4.3%) in ‘Medicine, Research and Experimental Pharmacology and Pharmacy’, 25 (4.0%) in ‘Cardiac and Cardiovascular Systems’, 22 (3.5%) in ‘Medicine, General and Internal’, 19 (3.0%) in ‘Psychiatry’, and 16 (2.5%) in ‘Geriatrics and Gerontology’. For 14 (2.2%) articles, the authors could not find the scientific field of the journals in the Web of Science (Table 4 all journal categories are represented in Supplementary table S3; published articles per journal title are represented in Supplementary table S4).

Major health topics included cardiovascular (181 trials, 28.8%), prevention (106 trials, 16.9%), mental health (83 trials, 13.2%), musculoskeletal (67 trials, 10.7%), respiratory (32 trials, 5.1%), infections (29 trials, 4.6%), neurological (22 trials, 3.5%), rational prescribing (18 trials, 2.9%), women’s health or maternal health (17 trials, 2.7%), chronic care management (15 trials, 2.4%), alcohol or substance use (14 trials, 2.2%), quality management or quality improvement (9 trials, 1.3%), elderly care (7 trials, 1.1%), nephrology (6 trials, 1.0%), and cancer (5 trials, 0.8%). Health topics with fewer than five trials each included dental (3 trials); malnutrition (3 trials); dermatology (2 trials); ear, nose and throat diseases (2 trials); pressure ulcers (2 trials); blood donation (1 trial); foot problems (1 trial); gastroenterology (1 trial); occupational health (1 trial); and urological (1 trial).

Table 1:  Characteristics of eligible studiestable image

Table 2:  Number of published protocols per registrytable image

Table 3:  Top 10 countries based on number of published protocolstable image

Table 4:  Top 10 journal categories based on number of published protocolstable image

Supplementary table S1: Funding source per geographical region

Supplementary table S2: Number of published protocols per country

Supplementary table S3: Number of published protocols per journal category

Supplementary table S4: Number of published protocols per journal

Discussion

A compilation of published protocols of RCTs in PHC was systematically created. Almost all publications appeared in the literature after 2000; half of the articles were published during the previous 10 years. The majority of articles described protocols about studies conducted in Western Europe, North America, and the Mediterranean. More than half of the protocols were conducted in the UK, the Netherlands, and the USA. Spain published the highest number of protocols among all Mediterranean countries. Articles often reported the funding source for the study. Few protocols stated that they were funded by industry, either in combination with a public or a non-governmental institute, or as the only source. The vast majority of protocols were indexed in a clinical study database; two registries (ISRCTN Registry and ClinicalTrials.gov) included more than two-thirds of the protocols.

Searching for RCT protocols in PHC was a challenging task. Family medicine or general practice may be the main provider for PHC in the majority of countries. However, PHC often serves as an umbrella term that could also include nursing, midwifery, psychology support, and physiotherapy services13. The extent to which a healthcare professional provides services within a PHC setting was not always clear in the screened studies14-18; in several cases provision of these services in a highly specialized setting could not be excluded. Therefore, unless investigators explicitly stated that both population and interventions were placed in PHC, the article was not considered as eligible. PHC settings are quite diverse among countries, regions, and healthcare systems despite the fact that they may have certain principles in common6,19. This is especially obvious when settings in rural areas are considered20-23. For articles οn PHC, investigators need to sufficiently explain the setting in their research protocol to avoid an erroneous exclusion, or misinterpretation of their results24.

According to the present study, the publication of RCT protocols οn PHC appeared during the last two decades, despite the fact that data since the 1990s supported that the publication and review of a research protocol can enhance the quality of studies25. This may be in accordance with high impact peer review journals, such as the Lancet in 1997, and the British Medical Journal in 2005, which strongly recommended publishing RCT protocols26,27 as well as with initiatives by scientific organizations to provide investigators with a research agenda for PHC or by funding opportunities that support PHC in specific countries28-32. In addition, the authors noted that a significant proportion of published protocols were designed in specific geographic regions and countries. Designing the protocol for an RCT requires a well-established infrastructure, a human capacity building environment, and adequate resources to support the implementation of the trial. A possible explanation is that countries such as the UK, the Netherlands, the USA, Spain, and Australia may satisfy these prerequisites in order to publish more than half of the eligible protocols on PHC33. The present study’s finding is in accordance with previous recommendations for the necessity of research capacity building in PHC among the non-prolific countries34,35. Under-representativeness of several countries in RCT published protocols may also explain for the potential lack of high-quality evidence for the effectiveness of interventions in the PHC settings of these countries. Similar findings have been previously described in other fields as well36,37. Rural PHC is a field that specifically lacks well-designed RCTs38. Acquiring adequate evidence to support clinical guidance on PHC should become a priority among non-prolific countries and areas while sufficient funding needs to support endeavors towards research equity worldwide.

The present study raised a few interesting observations while searching for protocols on PHC. The search in electronic databases other than PubMed yielded no additional published protocols; searching the PubMed database seemed to be adequate for identifying published RCT protocols on PHC. A very high percentage of these protocols were indexed. This may be in accordance with the latest instructions by several peer-reviewed journals that only registered protocols would be acceptable for publication39. Two registries (ISRCTN Registry and ClinicalTrials.gov) included more than 70% of the protocols. This may reflect the regions Western Europe and North America, which mainly contributed to the publications. It may also encourage future stakeholders who wish to find RCT protocols on PHC to start their search from these two registries. Finally, protocols were published in journals of several scientific fields; thus, investigators who are in quest of PHC protocols need to expand future searches beyond journals that belong to the field of ‘Primary Health Care’.

Published protocols invariably reported that they received funding. The majority of the RCTs received governmental or public support while industry participated in fewer than 8% of the studies. This was consistent across different geographical regions. A possible explanation for the limited participation of industry might be the type of interventions assessed in these protocols. PHC usually includes complex interventions that are not necessarily based on pharmaceutical components40. Research agenda in PHC is more likely to evaluate interventions aiming at improving access of care in remote areas, including rural health care, promoting healthy behaviors for prevention, managing multiple morbidity and frailty, addressing patients with multiple medications, supporting self-care, and providing palliative care41. These interventions do not exclude a pharmaceutical component. However, the main stakeholders include healthcare professionals who may be requested to implement effectively a complex intervention; the state, which may need to include the intervention in the regular budget; and the patients, who may benefit from an effective action42. This may deprive research in PHC from a sponsor such as industry. On the other hand, it implies that, in PHC research, if we need to search for explanations of potential caveats, we have to primarily investigate clinical, methodological, and implementation issues rather than a potential influence of industry.

This work is not strictly relevant to the health of rural and remote communities. However, its findings including the recent publication of protocols, the unrepresentativeness of several countries and regions and the sparse funding from industry may all characterize clinical trials in rural and remote communities as well. The authors raised issues related to search strategies for identifying PHC trial protocols and reporting of PHC settings that may advance knowledge essential to understanding and improving the design of clinical trials in PHC including the care of rural and remote communities.

The present study had several limitations. There were several articles for which the study’s relevance to PHC was ambiguous. All corresponding authors of the articles that needed clarification were contacted. However, several investigators did not reply. In addition, articles that might have been related to PHC but did not use any of the search terms to describe the setting may have not been captured. Thus, a few articles related to PHC without clearly naming it may have been left out. In addition, because the authors’ last search was on January 2015, several recently published protocols have not been included. To briefly explore whether there was any significant differences between them and the ones that have been already included in this article, the authors extracted the characteristics of 50 protocols on PHC published between January 2015 and June 2019 (Supplementary table S5). As in the present study sample, recent protocols were registered, reported trials that would be conducted in Central and Western Europe as well as in Northern America, and received government or public funding (Supplementary table S5). The present study was based on published RCT protocols only; therefore, RCT protocols that have never been published in a peer-reviewed journal were not identified. Thus, the actual number of ongoing or completed RCT protocols in PHC worldwide may have been underestimated. In addition, the present study database was restricted only to RCT protocols; therefore, the results cannot be generalized for other study designs in PHC. Interventions that involved both PHC and specialized care were excluded. Therefore, results may not be generalizable for interventions based on the collaboration of multiple healthcare levels (ie multidisciplinary and transition-related interventions). Finally, the results may not be generalizable for protocols of pilot or feasibility studies on PHC as well as for RCT protocols not published in English.

Supplementary table S5: Characteristics of eligible studies published between January 2015 and June 2019

Conclusion

Despite the previous limitations, a compilation of published RCT protocols was created based on the explicit statement of the investigators that the population and intervention for their study were placed in a PHC setting. To facilitate future searches for RCT protocols on PHC, investigators need to decide on specific criteria that would make the PHC setting easily identifiable. These criteria may include the explicit description of potential variations in usual care, resources needed in PHC, factors that may affect access in PHC, legal aspects that may intervene in the implementation, and structural issues for PHC that may interfere with policy making processes. In addition, supporting publication of study protocols on PHC, either by encouraging prospective indexing of trial protocols through registries, or by building research capacity in non-prolific countries and areas, including rural and remote regions, may further enhance research quality, collaboration, and evidence translation in PHC.

references:

1 Denegri S, Faure H. It’s plain and simple: transparency is good for science and in the public interest. Trials 2013; 14: 215. https://doi.org/10.1186/1745-6215-14-215 PMid:23849479
2 Treweek S, McCormack K, Abalos E, Campbell M, Ramsay C, Zwarenstein M, et al. The Trial Protocol Tool: the PRACTIHC software tool that supported the writing of protocols for pragmatic randomized controlled trials. Journal of Clinical Epidemiology 2006; 59: 1127-1133. https://doi.org/10.1016/j.jclinepi.2005.12.019 PMid:17027422
3 Hysong SJ, Woodard LC, Garvin JH, Murawsky J, Petersen LA. Publishing protocols for partnered research. Journal of General Internal Medicine 2014; 29: 820-4. https://doi.org/10.1007/s11606-014-3037-0 PMid:25355092
4 Wilson PM, Sales A, Wensing M, Aarons GA, Flottorp S, Glidewell L, et al. Enhancing the reporting of implementation research. Implementation Science 2017; 12: 13. https://doi.org/10.1186/s13012-017-0546-3 PMid:28178987
5 Ebell MH, Sokol R, Lee A, Simons C, Early J. How good is the evidence to support primary care practice? Evidence Based Medicine 2017; 22: ebmed-2017-110704. https://doi.org/10.1136/ebmed-2017-110704 PMid:28554944
6 Silagy C. Developing a register of randomised controlled trials in primary care [Erratum in British Medical Journal 1993;306(6893):1660]. British Medical Journal 1993; 306: 897-900. https://doi.org/10.1136/bmj.306.6882.897 PMid:8490419
7 Knottnerus JA. Research in general practice. Lancet 1996; 347: 1236-1238. https://doi.org/10.1016/S0140-6736(96)90745-X
8 Van Weel C. Primary care: political favourite or scientific discipline? Lancet 1996; 348: 1431-1432. https://doi.org/10.1016/S0140-6736(96)08116-0
9 Knottnerus JA. Between iatrotropic stimulus and interiatric referral: the domain of primary care research. Journal of Clinical Epidemiology 2002; 55: 1201-1206. https://doi.org/10.1016/S0895-4356(02)00528-0
10 Glanville J, Kendrick T, McNally R, Campbell J, Hobbs FD. Research output on primary care in Australia, Canada, Germany, the Netherlands, the United Kingdom, and the United States: bibliometric analysis. British Medical Journal 2011; 342: d1028-d1028. https://doi.org/10.1136/bmj.d1028 PMid:21385804
11 Brown L, Carne A, Bywood P, McIntyre E, Damarell R, Lawrence M, et al. Facilitating access to evidence: primary health care search filter. Health Information and Libraries Journal 2014; 31: 293-302. https://doi.org/10.1111/hir.12087 PMid:25411047
12 Gill PJ, Roberts NW, Wang KY, Heneghan C. Development of a search filter for identifying studies completed in primary care. Family Practice 2014; 31: 739-745. https://doi.org/10.1093/fampra/cmu066 PMid:25326923
13 Pols DH, Bramer WM, Bindels PJ, van de Laar FA, Bohnen AM. Development and validation of search filters to identify articles on family medicine in online medical database. Annals of Family Medicine 2015; 13: 364-366. https://doi.org/10.1002/9781119085621.wbefs137
14 Ickenroth MH, Janaica Grispen JE, de Vries NK, Dinant GJ, Elwyn G, Ronda G, et al. A single-blind randomised controlled trial of the effects of a web-based decision aid on self-testing for cholesterol and diabetes. Study protocol. BMC Public Health 2012; 12: 2-9. https://doi.org/10.1186/1471-2458-14-921 PMid:25194420
15 van de Ven G, Draskovic I, Adang EM, Donders RA, Post A, Zuidema SU, et al. Improving person-centred care in nursing homes through dementia-care mapping: design of a cluster-randomised controlled trial. BMC Geriatrics 2012; 12: 1. https://doi.org/10.1186/1471-2318-12-1 PMid:22214264
16 Bonevski B, Paul C, D’Este C, Sanson-Fisher R, West R, Girgis A, et al. RCT of a client-centred, caseworker-delivered smoking cessation intervention for a socially disadvantaged population. BMC Public Health 2011; 11: 70. https://doi.org/10.1186/1471-2458-11-70 PMid:21281519
17 Coleman T, Thornton J, Britton J, Lewis S, Watts K, Coughtrie MW, et al. Protocol for the smoking, nicotine and pregnancy (SNAP) trial: double-blind, placebo-randomised, controlled trial of nicotine replacement therapy in pregnancy. BMC Health Services Research 2007; 7: 2. https://doi.org/10.1186/1472-6963-7-2 PMid:17201904
18 Coleman T, McEwen A, Bauld L, Ferguson J, Lorgelly P, Lewis S. Protocol for the Proactive Or Reactive Telephone Smoking CeSsation Support (PORTSSS) trial. Trials 2009; 10: 26. https://doi.org/10.1186/1745-6215-10-26 PMid:19400961
19 Keogh C, Wallace E, O’Brien KK, Galvin R, Smith SM, Lewis C, et al. Developing a web-based international register of clinical prediction rules for primary care. SAPC ASM, University of Nottingham 2013; 12: 359-366. https://doi.org/10.1370/afm.1640 PMid:25024245
20 Aaraas IJ, Swensen E. National Centre of Rural Medicine in Norway: a bridge from rural practice to the academy. Rural and Remote Health 2008; 8: 948. Available: http://www.rrh.org.au/journal/article/948 (Accessed 20 January 2020).
21 Lionis C, Symvoulakis EK, Vardavas CI. Implementing family practice research in countries with limited resources: a stepwise model experienced in Crete, Greece. Family Practice 2010; 27: 48-54. https://doi.org/10.1093/fampra/cmp078 PMid:19884125
22 McGrail MR, Jones R, Robinson A, Rickard CM, Burley M, Drysdale M. The planning of rural health research: rurality and rural population issues. Rural and Remote Health 2005; 5: 426. Available: http://www.rrh.org.au/journal/article/426 (Accessed 20 January 2020).
23 Robinson A, Burley M, McGrail MR, Drysdale M, Jones R, Rickard CM. The conducting and reporting of rural health research: rurality and rural population issues. Rural and Remote Health 2005; 5: 427. Available: http://www.rrh.org.au/journal/article/427http://www.rrh.org.au/journal/article/427 (Accessed 20 January 2020).
24 Missiou A, Tatsioni A. Systematic reviews do not comment on applicability for primary care. Journal of Clinical Epidemiology 2015; 68: 1152-1160. https://doi.org/10.1016/j.jclinepi.2015.06.003 PMid:26093310
25 Altman DG. The scandal of poor medical research: a possible shortfall of qualified medical school applicants. BMJ 1994; 308: 283-284. https://doi.org/10.1136/bmj.308.6924.283 PMid:8124111
26 McNamee D, James A, Kleinert S. Protocol review at The Lancet. Lancet 2008; 372: 189-190. https://doi.org/10.1016/S0140-6736(08)61053-3
27 Jones G, Abbasi K. Trial protocols at the BMJ. BMJ 2004; 329: 1360. https://doi.org/10.1136/bmj.329.7479.1360 PMid:15591544
28 Hummers-Pradier E, Beyer M, Chevallier P, Eilat-Tsanani S, Lionis C, Peremans L, et al. The Research Agenda for General Practice/Family Medicine and Primary Health Care in Europe. Part 1. Background and methodology. European Journal of General Practice 2009; 15: 243-250. https://doi.org/10.3109/13814780903452184 PMid:20055723
29 Sullivan F, Butler C, Cupples M, Kinmonth AL. Primary care research networks in the United Kingdom. British Medical Journal 2007; 334: 1093-1094. https://doi.org/10.1136/bmj.39190.648785.80 PMid:17525452
30 Rortveit G. Research networks in primary care: an answer to the call for better clinical research. Scandinavian Journal of Primary Health Care 2014; 11: 1-3. https://doi.org/10.3109/02813432.2014.943981 PMid:25110911
31 Cooper A, Chuter A. Patient safety research in primary care: where are we now? British Journal of General Practice 2015; 65: 622-623. https://doi.org/10.3399/bjgp15X687709 PMid:26622010
32 Rørtveit G. Primary care research in Norway: going in the right direction. Scandinavian Journal of Primary Health Care 2017; 35: 1-2. https://doi.org/10.1080/02813432.2017.1288818 PMid:28277050
33 Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Quarterly 2005; 83: 457-502. https://doi.org/10.1111/j.1468-0009.2005.00409.x PMid:16202000
34 Lionis C, Allen J, Sapouna V, Alegakis A, Svab I. Towards the establishment of a new target strategy for WONCA Europe: a key informants study. European Journal of General Practice 2008; 14: 76-82. https://doi.org/10.1080/13814780802410148 PMid:18932109
35 Godue C, Cameron R, Borrell RM. Capacity building in human resources for health: the experience of the region of the Americas. Canadian Journal of Public Health 2016; 107: 347. https://doi.org/10.17269/CJPH.107.5564 PMid:28026696
36 Kelaher M, Ng L, Knight K, Rahadi A. Equity in global health research in the new millennium: trends in first-authorship for randomized controlled trials among low and middle-income country researchers 1990–2013. International Journal of Epidemiology 2016; 45: 2174-2183. https://doi.org/10.1093/ije/dyw313 PMid:28199646
37 Kalra A, Goindi S. Issues impacting therapeutic outcomes in pediatric patients: an overview. Current Pediatric Reviews 2014; 10: 184-193.
38 Lionis C, Tatsioni A. Conducting research in rural primary care medicine: do we need more experimental research or guidance? Rural and Remote Health 2012; 12(4): 2267. Available: http://www.rrh.org.au/journal/article/2267 (Accessed 20 January 2020).
39 De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Arteriosclerosis, Thrombosis, and Vascular Biology 2005; 25: 873-874. https://doi.org/10.1161/01.ATV.0000162428.48796.22 PMid:15790941
40 Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care-effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open 2015; 5: e009993. https://doi.org/10.1136/bmjopen-2015-009993 PMid:26700290
41 Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008; 337: 979-983. https://doi.org/10.1136/bmj.a1655 PMid:18824488
42 Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implementation Science 2013; 8: 139. https://doi.org/10.1186/1748-5908-8-139 PMid:24289295

supplementary content:

Supplementary table S1: Funding source per geographical regiontable image

Supplementary table S2: Number of published protocols per countrytable image

Supplementary table S3: Number of published protocols per journal categorytable image

Supplementary table S4: Number of published protocols per journaltable image

Supplementary table S5: Characteristics of eligible studies published between January 2015 and June 2019table image

appendix I:

Appendix I: Search strategytable image