Validating competencies for an undergraduate training program in rural medicine using the Delphi technique

Introduction: Worldwide, half the population lives in rural or remote areas; however, less than 25% of doctors work in such regions. Despite the continental dimensions of Brazil and its enormous cultural diversity, only some medical schools in this country offer students the opportunity to acquire work experience focused on medicine in rural or remote areas. The objective of the present study was to develop a framework of competencies for a longitudinal medical training program in rural medicine as an integrated part of medical training in Brazil. Methods: Two rounds of a modified version of the Delphi technique were conducted. Initially, a structured questionnaire was elaborated, based on a literature review. This questionnaire was submitted to the opinion of 20 panelists affiliated with the Rural Medicine Working Party of the Brazilian Society of Family and Community Medicine. The panelists were asked to evaluate the relevance of the competencies using a five-point Likert-type scale. In this study, the consensus criterion for a competency to be included in the framework was it being deemed 'very important' or 'indispensable' by a simple majority of the participants, while the criterion for excluding a competency was that a simple majority of the panel members considered that it 'should not be included' or was 'of little importance'. When a consensus was not reached regarding a given competency, it was submitted to a second round to enable the panelists to re-evaluate the now dichotomized questions. Results: Compliance in responding to the questionnaire was better among the panelists predominantly involved in teaching activities (85%; n=12) compared to those working principally in patient care (45%; n=8). The questionnaire consisted of 26 core


Introduction
Inequality with respect to the geographic distribution of health professionals is a global issue confronting various countries, irrespective of their level of socioeconomic development.While half the world's population lives in rural or remote areas, less than 25% of doctors work in such regions 1 .
The dearth of these professionals in rural areas is a multifactorial problem, related to professional isolation, lack of safety in the workplace, low salaries, irregular remuneration, lack of career advancement perspectives, difficult access to continued education, and lack of structure in the work environment and in the community [2][3][4][5] .For this reason, three retention domains should be considered: professional, personal and community.
Nevertheless, some educational interventions are strongly associated with physician attraction and retention in rural and remote areas.These include rurally located medical schools, multifaceted education interventions, continuing medical education and rurally relevant curricula and rural clinical placements [6][7][8][9][10][11] .Despite this evidence, the Flexnerian model still largely prevails in schools of medicine 12 , and the majority of universities give little or no value to experiences acquired in rural learning environments.Most undergraduate clinical practice still takes place in large cities and in tertiary hospitals, sometimes with outmoded and fragmented curricula.The disparities between the competencies developed during undergraduate training and the needs of the population come, therefore, as no surprise.Hence, a reform is needed that would transform the learning experience by adopting a line of teaching based on competencies with the community in mind.This would lead to a new model of professional with a strong sense of social responsibility [13][14][15] .
There are few data in the literature on the competencies that should be developed by an undergraduate student in a rural medicine program 16 .Despite the continental dimensions of Brazil and its enormous cultural diversity, only some medical schools in this country offer students the opportunity to acquire work experience focused on rural or remote areas, and to the best of the authors' knowledge there is no record of any longitudinal program in rural medicine or of an undergraduate competency-based curriculum that corresponds to the needs of rural populations [17][18][19][20][21] .Therefore, the objective of the present study was to elaborate and validate a framework of competencies in rural medicine to be developed as an integrated part of medical training.

Methods
This study applied a modified version of the Delphi technique to enable specialists to reach a consensus on the relevant competencies required for a longitudinal program of rural medicine.Thirty-eight doctors affiliated with the Rural Medicine Working Party Group of the Brazilian Society of Family and Community Medicine were invited to make up a panel of specialists.Data were collected in February and March 2015 at the Faculdade Pernambucana de Saúde (Pernambuco Faculty of Health) using an online questionnaire.
The selection criterion for the inclusion of the physicians in the specialist panel was that they had to be affiliated with the Family Medicine and Rural Community Group of the Brazilian Society of Family and Community Medicine, since this is the only group in Brazil that is recognized and structured for dealing with studies on this subject.The selected physicians were included in the study after agreeing to and signing an informed consent form.At the time of data collection, the Rural Medicine Working Party consisted of 40 doctors distributed throughout the various regions of the country.Fifteen physicians also worked as medical professors or preceptors, three worked as health managers and 22 worked principally in patient care.Two of them were involved with the study and for ethical reasons were not included on the panel of specialists.
For the first step in this study, a structured, five-point Likerttype questionnaire was put together with the following possible options of response: 'should not be included', 'of little importance', 'relatively important', 'very important' and 'indispensable'.This instrument, composed of questions regarding competencies in rural medicine, was elaborated following a literature review.The terms rural health, competence and medical education were used to search for articles in the Medline, CINAHL, Academic Search Alumni Edition, Education Research Complete, Education Resources Information Center (ERIC) and PUBMED databases.Studies dealing with the subject in question were selected and those defining or suggesting the competencies required for working in health care in rural areas were used to construct the first questionnaire.In addition, the online curriculum frameworks of some international institutes 22 that included structured competency-based work experience in medicine or rural health and the rural populations health status were assessed, with the objective of identifying the competencies that best meet the needs of this segment of the population.
With the exclusion of competencies referring specifically to settings and situations in other countries that were not relevant to this particular setting, and to focal specialties (ie those restricted to a certain gender, age group, system or organ), the competencies found were used by the principal investigator of this study to construct the questionnaire.The competencies that according to the literature were most relevant, as well as the more general competencies, sometimes subdivided into other competencies, were classified as core competencies.The competencies contained within the core competencies, those with more specific characteristics, were classified as secondary competencies.The resulting instrument consisted of a three-page questionnaire, with space for suggesting new competencies and for comments at the end of each page.Its objective was to serve as a foundation on which to build the framework.
The first Delphi round consisted of sending the instrument to the panel of specialists to allow them to evaluate the relevance of each one of the competencies.In the second round, a new online questionnaire was sent to the members of the Rural Medicine Working Party who had agreed to participate in the study.After examining the results of the previous round, the panelists re-evaluated those competencies for which no consensus had been reached in the first round and evaluated the suggestions that had been made.Data were collected using an online research tool, Survey Monkey.The study participants were registered in the software program by their email addresses, which were obtained from the website of the Rural Medicine Working Party or by telephone contact.In both rounds, the investigators used the program to send the instrument simultaneously to all the participants, together with a letter inviting them to participate in the study and including instructions on how to complete and return the questionnaire.The invitation letter included a protected link through which the recipient was able to gain access to the informed consent form.After agreeing to the terms of the informed consent form, the participant was then able to access the questionnaire.In both rounds, the participants were given a 15-day period in which to answer the questionnaire, with reminders being sent periodically.
The data were analyzed using spreadsheet software and Epi Info v3.5 (Centers for Disease Control and Prevention; https://wwwn.cdc.gov/epiinfo/html/prevVersion.htm).Descriptive measures such as means, 25th and 75th percentiles, and percentages of relevant (mean rate >3.0) and non-relevant scores were calculated for the responses to each item in the questionnaires.The consensus criterion established for a competency to be included in the framework was it being deemed 'very important' or 'indispensable' by a simple majority of the participants, while the criteria for excluding a competency was that a simple majority of the panel considered that it 'should not be included' or was 'of little importance'.The competencies that failed to reach a consensus were submitted to a second round of the Delphi method to allow the panelists to re-evaluate them, now in the form of dichotomized questions 23 .In this way, more competencies could be evaluated in the second round for possible inclusion in the framework.

Results
The literature review identified 95 articles related to the topic, 15 of which listed competencies in rural medicine and were used by the principal investigator of this study to construct the first-round questionnaire.This consisted of 26 core competencies and 165 secondary competencies, which, in accordance with the multidimensional aspect of the professional competency, were later grouped into six domains: cognitive, technical, affective, relational, integrative and contextual, as proposed by Epstein and Hundert 24 .Adhesion to the study was 52% (n=20): 80% (12/15) among medical professors and preceptors, 100% (3/3) among the health managers and 45% (9/20) among those working predominantly in patient care.In the first round of the Delphi technique, consensus was obtained for the 26 core competencies.Of these, 26.9% (n=7) were evaluated by 80% or more of the participants (n=16) as 'indispensable' competencies: • 'to communicate effectively with the sick person, the community and the healthcare team' • 'to show empathy for the patient and for the community' • 'to show a high level of clinical perspicacity in performing diagnoses in a setting in which supplementary tests are rare and access to a specialist is difficult' • 'to be able to perform necessary procedures in a setting where access to a specialist is difficult' • 'to make home visits decisively, ethically and compassionately' • 'to use technology to communicate and exchange information with colleagues in distant locations for the purpose of continued education and specialist orientation' • 'to schedule patient consultation times with a view to meeting demand effectively and equitably, taking into consideration the particular difficulties and challenges of the geographical, cultural and socioeconomic contexts of rural patients' (Tables 1,2).
Of the secondary competencies, only eight failed to obtain a consensus.The majority of these were related to hospital procedures and transportation (Table 3).
Although various suggestions were made, all of them referred to items that had already been included in the questionnaire and no new competency was suggested.Nevertheless, it was proposed that rural medical training in Brazil should be provided within the specialization program in Family and Community Medicine, since medical training cannot be restricted only to the undergraduate medical course.The need for specialization to enable a doctor to work in primary healthcare was emphasized.Fifty percent of the physicians (n=10) completed the secondround questionnaire: seven teachers (70%) and three (30%) physicians working principally in patient care.In this round, the specialists evaluated a new instrument containing dichotomized questions on those competencies that had not achieved a consensus in the first round.Of the competencies that failed to achieve a consensus in the first round, 87.5% (n=7) were excluded from the framework in the second round.There was a consensus (100%; n=10) with respect to the proposal made in the first round regarding training in rural medicine.The final framework consisted of 26 core competencies and 158 secondary competencies.

Discussion
In line with the transformation occurring in medical education worldwide, this study constructed and validated a framework of competencies for a longitudinal program of rural medicine, with a view to training physicians who will have a sense of social responsibility and a greater inclination and readiness to meet the needs of rural populations 13 .
The response rate of 52% achieved in the first round was within the expected limits for the Delphi method 25 .Adherence was found to be greater among the doctors working predominantly in teaching and among the managers, a fact that may reflect the greater motivation of these professionals in relation to the subject matter compared to those working predominantly in patient care.A modified version of the Delphi technique was selected, with the open questions of the classic first round being substituted by a structured questionnaire constructed following a literature review.The objective of using this method was to maximize the response rate in the first round, ensuring the inclusion of important topics that could otherwise have been omitted by the specialists and increasing the likelihood of achieving a consensus in a more effective manner, as already suggested by other authors 26,27 .Consequently, two rounds were sufficient for obtaining a consensus, as previously shown in a similar study 28 .With the Delphi method, it was also possible to connect panelists distributed throughout the various regions of the country.This not only rendered the study viable but also introduced a broader perspective to the discussion regarding the particular competencies required for doctors working in different rural settings.A study conducted in 2004 in Australia systematized the principal differences between general practitioners working in rural and urban areas through a review of the pertinent literature.It was found that rural physicians, by being isolated geographically, with few available resources and limited access to specialists, needed to be more decisive, self-reliant and independent.They worked longer hours and were often required to have skills used in other specialties such as anesthesia, surgery and emergency medicine in addition to having to deal with hospitalized patients.They needed to be highly skilled with respect to the diagnosis, treatment and clinical management of health problems, and had to know how to behave in an intercultural context 29 , unlike other doctors working in urban environments, who were more easily able to refer their patients to specialists and had ready access to complementary tests.
The competencies deemed relevant by the specialists reflected the principal needs of rural communities.The seven most relevant competencies, evaluated as indispensable by 80% of the specialists, were related principally to communication, empathy, clinical perspicacity, performing procedures, home visits, continued education and planning.
Users' perceptions in relation to the quality of the healthcare services is strongly associated with tangible elements of the interaction between the healthcare professional and the patient, generally related to aspects of communication and expressions of empathy 30 .Involvement with the patient through responsive attitudes to his/her concerns increases user satisfaction 31 .Other investigators have emphasized the importance of knowledge on sociohistorical and cultural aspects in improving understanding of the problems experienced by these populations and the direct or indirect effect of these issues on their health conditions 32 .
The relevance of clinical perspicacity and skills in performing procedures lies in meeting the needs of the individuals living in these areas, consequently resolving their health issues within or close to their communities.Some specialists suggest that centers equipped with transportation would solve these problems.However, this is controversial, with other professionals arguing that this alternative could be applied to more specific problems requiring, for example, specialist surgeries, but not for routine problems 33 .For this reason, in a setting in which access to specialist services and supplementary tests is difficult, clinical perspicacity becomes even more important as well as the ability to reach diagnosis based on an accurate analysis of signs and symptoms; perform locoregional anesthesia, resection of lipomas and nevi; and to drain abscesses, etc.There is evidence that students who spend time in rural areas tend to develop better clinical skills 34,35 .
The lack of healthcare professionals trained to attend child birth is a major problem in rural areas.The dearth of these professionals increases the number of infants born without adequate care and of women who have to travel to give birth in other municipalities 36 .A study conducted in Ecuador highlighted gaps in the obstetric training of recent graduates in relation to the needs of rural populations, emphasizing the need to prepare them to attend normal childbirth, to repair perineal lacerations and to perform neonatal resuscitation 37 , competencies that have also been included in the framework developed in the present study.
Studies indicate that rural doctors carry out more home visits, attend more weekend consultations and call-outs, and cover greater distances than their colleagues working in urban areas 38 .For these reasons, students have to be trained to carry out home visits effectively.It is important that they know how to manage the time and resources available to allow them to meet demands equitably.The need for students to acquire knowledge on palliative care should also be emphasized, since individuals living in rural areas also report greater difficulty in accessing this type of care, and lack of knowledge in this area is a common complaint of rural doctors who feel unprepared to provide it 39 .
The use of technology for communication and for exchanging information with colleagues in other locations for the purpose of continued education and matrix development was another competency evaluated as indispensable by the panelists.Technology has become an extremely important tool with which to manage the professional isolation experienced by doctors in rural and remote areas.Telemedicine has brought advances to rural medicine, since it permits distant specialist learning with theoretical material and video classes, discussion forums, interactive consultations and specialist orientation 40 .It is a strategy that has been adopted to supervise students in rural areas 41 , and by government agencies to provide support to professionals working in these regions 42 .
The group considered that medical training is not an end in itself and that specialization is required for working in primary care.Therefore, rural doctors in Brazil should complete the training provided by the residency program in family and community medicine.It should be taken into consideration that the group of panelists consisted in its majority of family doctors and that, in accordance with the new national guidelines, recently graduated doctors should be qualified to work at any healthcare level 43 .
Nevertheless, the consensus of the panelists is in agreement with the idea that primary health care is not synonymous with simple problems, but rather involves complex problems requiring additional training in order to acquire specific skills 44 .Evidence indeed points to the fact that doctors training within the residency program in family and community medicine are those most likely to work with rural and remote populations 45 .In some countries, the team of rural doctors includes other specialties such as surgery and medical emergencies and indeed there is an ongoing debate in the literature regarding whether rural medicine should be a separate discipline 29 .
Evidence shows the importance of interactions between primary, secondary and tertiary care in enabling students to develop skills in hospital procedures 46 .However, in the present study, the competencies evaluated as irrelevant were mostly related to hospital procedures, reflecting the training of rural doctors in Brazil, which is predominantly related to primary care.

Conclusions
Considering the various possible ways of providing training in rural medicine, it is important to adopt a training model that meets the needs of the community and of the local healthcare system, and that contemplates the full extent of rural practice 47 .The debate on the terminality of the undergraduate medical course is extremely broad-reaching and topical; however, it was not one of the objectives of the present study.
The advances in the field of education in rural medicine as a subspecialty in Brazil are in their infancy, with a competencybased curriculum for a third year in the family and community medicine residency program in rural medicine having only been recently published 48 .Among the competencies taken into consideration is the ability to perform surgeries in emergency situations as an advanced skill, indicating a greater interaction between the rural doctor and the hospital environment, as is the case in other countries.
This study elaborated and validated a framework of competencies for a longitudinal program of rural medicine that consists of 26 core competencies and 158 secondary competencies.This framework should contribute to the construction of competency-based curricula in rural medicine for use in medical training in Brazil.

Table 1 :
Analysis of consensus on the relevance of the core competencies required for an undergraduate course in rural medicine according to domain: technical, cognitive, relational, Recife, Brazil, 2015.Core competencies partially accepted had one or more secondary competencies that failed to achieve consensus in the first round.1, 'should not be included'.2, 'of little importance'.3, 'relatively important'.4, 'very important'.5, 'indispensable'. ¶

Table 2 :
Analysis of consensus on the relevance of the core competencies required for an undergraduate course in rural medicine according to domain: affective, integrative, contextual, Recife, Brazil, 2015.

Table 2 :
cont'd Core competencies partially accepted had one or more secondary competencies that failed to achieve consensus in the first round.1, 'should not be included'.2, 'of little importance'.3, 'relatively important'.4, 'very important'.5, 'indispensable'.Competency evaluated as indispensable by at least 80% of the panelists.¶ Core competencies partially accepted had one or more secondary competencies that failed to achieve consensus in the first round.1, 'should not be included'.2, 'of little importance'.3, 'relatively important'.4, 'very important'.5, 'indispensable'. ¶

Table 3 :
Analysis of secondary competencies required for an undergraduate course in rural medicine that failed to achieve a consensus in the first round according to core competency, Recife, Brazil, 2015.