Searching for published protocols of randomized controlled trials in primary health care: an empirical systematic approach

Introduction

Prospective reporting of clinical trial protocols – either in a trial registry or as a published article – has been encouraged in recent decades. Several proponents suggested that prospective reporting of trial design might enhance transparency and therefore improve accountability in the conduct and reporting of research¹. In addition, researchers can have access to the design of ongoing trials that may be relevant to their work. Furthermore, the general public and other stakeholders can be informed of ongoing trials that may influence their decisions and policies in the future^2-4.

Robust conclusions from randomized controlled trials (RCTs) are pivotal for primary health care (PHC) to support both clinical decision making and healthcare policy⁵. RCTs in PHC are not exempted from prospective reporting; their protocols may be indexed in clinical trial registries or as published articles before reporting the results⁶. However, the search for protocols of RCTs in PHC may be challenging and time consuming because there is no globally accepted algorithm on identifying studies with population and intervention relevant to PHC.

The characteristic position of PHC has been well noted^7,8. Previous studies have described essential spectrum differences between PHC and specialized, referred (secondary) care⁹. However, the diversity of clinical areas and infrastructure among different countries in PHC complicates the identification of protocols designed for this field⁶. Previous articles have developed and validated electronic search filters on family medicine or general practice studies^10-13. However, to the authors’ knowledge, there has been no systematic effort to specifically identify and record protocols of RCTs in this field. Systematic searches that may successfully identify protocols on PHC may shed light on the direction of current research in PHC, reveal potential gaps, structure PHC research agenda and guide future developing of research capacity.

The present study aimed at systematically identifying published RCT protocols in PHC. In addition, information about specific characteristics including publication year, country, prospective registration, funding, and publication sources was captured. This compilation of PHC trial protocols is part of an ongoing project on assessing pragmatic/explanatory continuum in the design of published RCT protocols on PHC.

Methods

Search strategy

Electronic databases were searched including PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Scopus from inception (1966 for PubMed, 1981 for CINAHL, and 1996 for Scopus) to December 2014 (last search January 2015). A search strategy was created for published RCT protocols on PHC including keywords related to family medicine/general practice and PHC combined with the Cochrane Collaboration search strategy for published RCTs. The final strategy is shown in Appendix I. A similar systematic search was conducted on CINAHL after excluding articles registered in MEDLINE. The final strategy is shown in Appendix I. Finally, Scopus database was used to enhance search results. The exact strategy is shown in Appendix I. In a Scopus search, after excluding articles registered in MEDLINE, the journals for the remaining articles checked; the articles were excluded if they had been published in journals indexed in PubMed and therefore were likely to have already been screened among the PubMed publications. All articles that were included in journals not indexed in PubMed were screened.

Three investigators (EP, EEN, and AT) independently performed a pilot screening based on the title and/or abstract for the first 500 items in PubMed to identify potential disagreements in the screening process. After settling potential discrepancies with consensus, one investigator (EP) completed the screening of all titles and/or abstracts in PubMed, CINAHL, and Scopus. A second investigator (EEN or AT) checked on the items that the first investigator could not decide upon. For items for which both investigators were not able to decide on their eligibility, the authors retrieved the full text of the article. Three investigators (EP, AC, and EN) independently screened articles in full text. A fourth investigator (AT) resolved any discrepancies. For those articles that investigators were not able to decide on their eligibility based on full text, the authors electronically contacted the article’s corresponding author to request additional information.

Inclusion criteria

Published protocols of RCTs that were relevant to PHC were considered as eligible. An article that reported the methodology of an RCT without including any part of the results was considered as a protocol. An article was considered relevant to PHC if the investigators clearly stated that they would recruit participants from PHC, general practice, or family medicine, and that PHC professional general practitioners or family physicians would be involved in the intervention. In case investigators referred either to a mixed population (PHC and secondary/tertiary care) or an intervention including a specialized part, the article was excluded. There were not any exclusion criteria about the type of professionals involved; the investigators just had to clearly mention that they worked in PHC. Protocols on pilot or feasibility trials as well as protocols that described non-randomized or pseudo-randomized studies were excluded. Only publications written in English were included. No limitations on publication time were set.

Data extraction

For each eligible article, the extracted items included PMID, first author’s name, publication year, and the country in which it would be conducted; whether the protocol was registered and, if it was, the database it was indexed in; the registration date, the registration number, the funding source using the exact wording in the publication; and the journal in which it was published and its field according to Web of Science. In addition, each country was categorized according to the world region. The following regions were used: Western Europe, North America, Mediterranean, Oceania, Central Europe, Scandinavia, Africa, East Asia, South Asia, Central and South America, and the Middle East. Based on the funding source, the studies were categorized in the following groups: government or public funding, funded by non-governmental organization or institute, mixed funding excluding industry, mixed funding including industry, and funded by industry only. Additional categories included the studies that clearly stated that they were not funded and the publications that did not mention funding at all. Finally, the health topic covered by each protocol was recorded.  

Two independent investigators (CA and EN) extracted the data. Discrepancies were resolved by consensus; where a consensus could not be reached, a third arbitrator (AT) participated in the final decision.

Statistical analysis

Data were presented as absolute numbers and frequencies for binary and categorical variables, and as medians with interquartile range (IQR) for continuous variables. All analyses were performed with the Statistical Package for the Social Sciences v21.0 (IBM; http://www.spss.com.au).

Ethics approval

The study did not involve human subjects, and thus it was exempt from documentation of informed consent and institutional review board approval. 

Results

Eligible articles

The search yielded a total of 26 571 items in PubMed, 268 items in CINAHL journals that were not indexed in MEDLINE, and 670 items in Scopus that were published in 24 journals not indexed in MEDLINE or PubMed. Based on the title and/or abstract, 25 739 articles retrieved in PubMed and all the items found in CINAHL or Scopus were excluded as not relevant, mainly because they were not protocols. Notably, there was no study protocol among the articles registered only in CINAHL or Scopus. Thus, 832 articles were evaluated in full text. Out of the 832 publications that were potentially eligible, 81 were excluded because they were not relevant to PHC; 61 articles because they referred to mixed population and/or intervention including secondary health care; 27 articles as pilot/feasibility studies; 14 because they were not RCTs; and 1 publication because it was not a protocol.

For 29 articles, their eligibility for PHC after reading the full text was unclear and therefore the corresponding author was contacted by e-mail. A reply was received from 19 authors; 10 out of 19 responded that their article was not relevant to PHC.

Thus, 184 out of the 832 potentially eligible publications were excluded. In addition, 10 out of the 29 articles for which a response from their corresponding authors was not received were excluded; and one additional article that could not be retrieved in full text.

Out of the 27 509 potentially relevant items in PubMed, CINAHL, and Scopus, the final database included 628 eligible published RCT protocols on PHC after excluding 26,677 items based on title and /or abstract as not relevant; and after excluding 203 articles based on full text screening, and one article, which could not be retrieved in full text (Fig1).

Characteristics of eligible published protocols

For the 628 eligible articles, median publication year was 2011 (IQR 2009–2013). Out of the 628 published protocols, 572 (91.1%), reported that they were prospectively registered in a database of clinical studies (Table 1). Out of the 628 protocols, 262 (41.7%) were designed in Western Europe, 117 (18.6%) in North America, 77 (12.3%) in the Mediterranean, 62 (9.9%) in Oceania, 42 (6.7%) in Central Europe, 33 (5.3%) in Scandinavia, and 10 (1.6%) in Africa. The authors found five protocols (0.8%) conducted in eastern Asia including China and Japan, 4 (0.6%) in South-East Asia, 4 (0.6%) in South Asia including India, 2 (0.3%) in Central and South America, and 2 (0.3%) in the Middle East. In addition, 8 (1.3%) protocols were conducted in several regions (Table 1). Out of the 628 articles, 408 (65.0%) articles reported that they were funded by the government; 76 studies (12.1%) were supported by non-governmental organizations or institutes; 64 (10.2%) studies reported funding both by the government and by non-governmental organizations or institutes; while 30 (4.8%) articles reported funding by several sponsors including industry; and 15 (2.4%) that they were exclusively funded by industry. In one protocol, authors clearly stated that there was no funding. In 35 (5.6%) articles, authors did not mention any information about the funding source (Table 1). When the funding sources per geographical region were checked, no differences from the overall pattern were observed (Supplementary table S1).

Out of the 571 published protocols that mentioned registration in a clinical study database, 245 (42.9%) were indexed in the International Standard Randomised Controlled Trials Number (ISRCTN) Registry, 209 (36.6%) in ClinicalTrials.gov, 53 (9.3%) in the Australian New Zealand Clinical Trials Registry, and 47 (8.2%) in the Netherlands National Trial Registry (Table 2). The 10 countries that published the highest number of protocols are presented in Table 3. Specifically, 141 (22.5%) articles were from the UK, 105 (16.7%) from the Netherlands, 93 (14.8%) from the USA and 69 (11.0%) from Spain (Table 3; all countries are represented in Supplementary table S2). Australia, Germany, Canada, Denmark, and Norway were also among the top 10 countries. Nine (1.4%) articles included protocols that were conducted in several countries.

Table 4 presents the 10 field categories according to Web of Science, which included most of the journals that published the 628 protocols. Specifically, 117 (18.6%) articles were published in journals included in the ‘Medicine, Research and Experimental’ field, 105 (16.7%) in ‘Health Care Sciences and Services’, 102 (16.2%) in ‘Public, Environmental and Occupational Health’, 69 (11.0%) in ‘Primary Health Care’, 38 (6.1%) in ‘Orthopedics Rheumatology’, 27 (4.3%) in ‘Medicine, Research and Experimental Pharmacology and Pharmacy’, 25 (4.0%) in ‘Cardiac and Cardiovascular Systems’, 22 (3.5%) in ‘Medicine, General and Internal’, 19 (3.0%) in ‘Psychiatry’, and 16 (2.5%) in ‘Geriatrics and Gerontology’. For 14 (2.2%) articles, the authors could not find the scientific field of the journals in the Web of Science (Table 4 all journal categories are represented in Supplementary table S3; published articles per journal title are represented in Supplementary table S4).

Major health topics included cardiovascular (181 trials, 28.8%), prevention (106 trials, 16.9%), mental health (83 trials, 13.2%), musculoskeletal (67 trials, 10.7%), respiratory (32 trials, 5.1%), infections (29 trials, 4.6%), neurological (22 trials, 3.5%), rational prescribing (18 trials, 2.9%), women’s health or maternal health (17 trials, 2.7%), chronic care management (15 trials, 2.4%), alcohol or substance use (14 trials, 2.2%), quality management or quality improvement (9 trials, 1.3%), elderly care (7 trials, 1.1%), nephrology (6 trials, 1.0%), and cancer (5 trials, 0.8%). Health topics with fewer than five trials each included dental (3 trials); malnutrition (3 trials); dermatology (2 trials); ear, nose and throat diseases (2 trials); pressure ulcers (2 trials); blood donation (1 trial); foot problems (1 trial); gastroenterology (1 trial); occupational health (1 trial); and urological (1 trial).

Supplementary table S1: Funding source per geographical region

Supplementary table S2: Number of published protocols per country

Supplementary table S3: Number of published protocols per journal category

Supplementary table S4: Number of published protocols per journal

Discussion

A compilation of published protocols of RCTs in PHC was systematically created. Almost all publications appeared in the literature after 2000; half of the articles were published during the previous 10 years. The majority of articles described protocols about studies conducted in Western Europe, North America, and the Mediterranean. More than half of the protocols were conducted in the UK, the Netherlands, and the USA. Spain published the highest number of protocols among all Mediterranean countries. Articles often reported the funding source for the study. Few protocols stated that they were funded by industry, either in combination with a public or a non-governmental institute, or as the only source. The vast majority of protocols were indexed in a clinical study database; two registries (ISRCTN Registry and ClinicalTrials.gov) included more than two-thirds of the protocols.

Searching for RCT protocols in PHC was a challenging task. Family medicine or general practice may be the main provider for PHC in the majority of countries. However, PHC often serves as an umbrella term that could also include nursing, midwifery, psychology support, and physiotherapy services¹³. The extent to which a healthcare professional provides services within a PHC setting was not always clear in the screened studies^14-18; in several cases provision of these services in a highly specialized setting could not be excluded. Therefore, unless investigators explicitly stated that both population and interventions were placed in PHC, the article was not considered as eligible. PHC settings are quite diverse among countries, regions, and healthcare systems despite the fact that they may have certain principles in common^6,19. This is especially obvious when settings in rural areas are considered^20-23. For articles οn PHC, investigators need to sufficiently explain the setting in their research protocol to avoid an erroneous exclusion, or misinterpretation of their results²⁴.

According to the present study, the publication of RCT protocols οn PHC appeared during the last two decades, despite the fact that data since the 1990s supported that the publication and review of a research protocol can enhance the quality of studies²⁵. This may be in accordance with high impact peer review journals, such as the Lancet in 1997, and the British Medical Journal in 2005, which strongly recommended publishing RCT protocols^26,27 as well as with initiatives by scientific organizations to provide investigators with a research agenda for PHC or by funding opportunities that support PHC in specific countries^28-32. In addition, the authors noted that a significant proportion of published protocols were designed in specific geographic regions and countries. Designing the protocol for an RCT requires a well-established infrastructure, a human capacity building environment, and adequate resources to support the implementation of the trial. A possible explanation is that countries such as the UK, the Netherlands, the USA, Spain, and Australia may satisfy these prerequisites in order to publish more than half of the eligible protocols on PHC³³. The present study’s finding is in accordance with previous recommendations for the necessity of research capacity building in PHC among the non-prolific countries^34,35. Under-representativeness of several countries in RCT published protocols may also explain for the potential lack of high-quality evidence for the effectiveness of interventions in the PHC settings of these countries. Similar findings have been previously described in other fields as well^36,37. Rural PHC is a field that specifically lacks well-designed RCTs³⁸. Acquiring adequate evidence to support clinical guidance on PHC should become a priority among non-prolific countries and areas while sufficient funding needs to support endeavors towards research equity worldwide.

The present study raised a few interesting observations while searching for protocols on PHC. The search in electronic databases other than PubMed yielded no additional published protocols; searching the PubMed database seemed to be adequate for identifying published RCT protocols on PHC. A very high percentage of these protocols were indexed. This may be in accordance with the latest instructions by several peer-reviewed journals that only registered protocols would be acceptable for publication³⁹. Two registries (ISRCTN Registry and ClinicalTrials.gov) included more than 70% of the protocols. This may reflect the regions Western Europe and North America, which mainly contributed to the publications. It may also encourage future stakeholders who wish to find RCT protocols on PHC to start their search from these two registries. Finally, protocols were published in journals of several scientific fields; thus, investigators who are in quest of PHC protocols need to expand future searches beyond journals that belong to the field of ‘Primary Health Care’.

Published protocols invariably reported that they received funding. The majority of the RCTs received governmental or public support while industry participated in fewer than 8% of the studies. This was consistent across different geographical regions. A possible explanation for the limited participation of industry might be the type of interventions assessed in these protocols. PHC usually includes complex interventions that are not necessarily based on pharmaceutical components⁴⁰. Research agenda in PHC is more likely to evaluate interventions aiming at improving access of care in remote areas, including rural health care, promoting healthy behaviors for prevention, managing multiple morbidity and frailty, addressing patients with multiple medications, supporting self-care, and providing palliative care⁴¹. These interventions do not exclude a pharmaceutical component. However, the main stakeholders include healthcare professionals who may be requested to implement effectively a complex intervention; the state, which may need to include the intervention in the regular budget; and the patients, who may benefit from an effective action⁴². This may deprive research in PHC from a sponsor such as industry. On the other hand, it implies that, in PHC research, if we need to search for explanations of potential caveats, we have to primarily investigate clinical, methodological, and implementation issues rather than a potential influence of industry.

This work is not strictly relevant to the health of rural and remote communities. However, its findings including the recent publication of protocols, the unrepresentativeness of several countries and regions and the sparse funding from industry may all characterize clinical trials in rural and remote communities as well. The authors raised issues related to search strategies for identifying PHC trial protocols and reporting of PHC settings that may advance knowledge essential to understanding and improving the design of clinical trials in PHC including the care of rural and remote communities.

The present study had several limitations. There were several articles for which the study’s relevance to PHC was ambiguous. All corresponding authors of the articles that needed clarification were contacted. However, several investigators did not reply. In addition, articles that might have been related to PHC but did not use any of the search terms to describe the setting may have not been captured. Thus, a few articles related to PHC without clearly naming it may have been left out. In addition, because the authors’ last search was on January 2015, several recently published protocols have not been included. To briefly explore whether there was any significant differences between them and the ones that have been already included in this article, the authors extracted the characteristics of 50 protocols on PHC published between January 2015 and June 2019 (Supplementary table S5). As in the present study sample, recent protocols were registered, reported trials that would be conducted in Central and Western Europe as well as in Northern America, and received government or public funding (Supplementary table S5). The present study was based on published RCT protocols only; therefore, RCT protocols that have never been published in a peer-reviewed journal were not identified. Thus, the actual number of ongoing or completed RCT protocols in PHC worldwide may have been underestimated. In addition, the present study database was restricted only to RCT protocols; therefore, the results cannot be generalized for other study designs in PHC. Interventions that involved both PHC and specialized care were excluded. Therefore, results may not be generalizable for interventions based on the collaboration of multiple healthcare levels (ie multidisciplinary and transition-related interventions). Finally, the results may not be generalizable for protocols of pilot or feasibility studies on PHC as well as for RCT protocols not published in English.

Supplementary table S5: Characteristics of eligible studies published between January 2015 and June 2019

Conclusion

Despite the previous limitations, a compilation of published RCT protocols was created based on the explicit statement of the investigators that the population and intervention for their study were placed in a PHC setting. To facilitate future searches for RCT protocols on PHC, investigators need to decide on specific criteria that would make the PHC setting easily identifiable. These criteria may include the explicit description of potential variations in usual care, resources needed in PHC, factors that may affect access in PHC, legal aspects that may intervene in the implementation, and structural issues for PHC that may interfere with policy making processes. In addition, supporting publication of study protocols on PHC, either by encouraging prospective indexing of trial protocols through registries, or by building research capacity in non-prolific countries and areas, including rural and remote regions, may further enhance research quality, collaboration, and evidence translation in PHC.