Increasing the enrolment of rural applicants to the faculty of medicine and addressing diversity by using a priority matrix approach to assign values to rural attributes

In an external review of the admissions process for the Faculty of Medicine, University of Manitoba, Canada, it was suggested that admissions policies be modified to increase the enrolment of students more likely to practise in rural locations, by selecting a cohort of students with attributes reflecting potential for rural practice. A broad-based Working Group devised a framework for scoring personal attributes reflecting a potential for living and working in rural areas. This framework, based on established characteristics reported in the literature, valued applicants who had rural connections, a history of rural employment, a history of rural community service, or a combination of these attributes. Relative weights for the attributes were determined using a priority matrix approach. Historic admissions data, comprising applicants’ rural origin (defined only by location of high school graduation), composite scores, and ranking, were reanalyzed to identify the magnitude of numerical constants that, when applied to composite scores, enhanced the relative ranking of eligible rural-origin applicants. This resulted in a hypothetical 29%-33% increase in the number of rural-origin students in incoming classes in those years. In the inaugural year of implementation of the policy and methodology, 60 admission offers (44.1%) were made to applicants with one or more rural attributes. Without adjustments, only 49 applicants with rural attributes (36%) would have been offered admission. This methodology resulted in a 22.4% increase in admission offers to applicants with rural attributes, and ushered in an incoming class that was more representative of the province’s


Introduction
Approximately half of the global population lives in a rural area, but these areas are served by less than a quarter of the total physician workforce 1 .While healthcare inequity is especially severe in low-income countries, even high-income countries experience shortage of health workers in remote and rural areas.
Approximately 20% of the populations in the USA and Canada live in rural areas 1,2 .The health status of the rural populations in both countries is thought to be lower than that of their urban counterparts 2,3 .Yet, only 9-10% of these nations' physicians serve the healthcare needs of the rural populations 1,2 .Shortages in the physician workforce have been reported in the rural and remote areas of the Canadian province of Manitoba 4 .In addition, specialist care is concentrated in Winnipeg and Brandon and, therefore, is not easily accessible to rural Manitobans 4 .The University of Manitoba operates under the social accountability vision adopted by all Canadian medical schools, and one principal aim is to improve access to physician services in rural Canada 5 .
Accordingly, the university reaffirmed its ongoing commitment to develop a sufficient and sustainable physician workforce, and re-examined its efforts to recruit, enroll and educate students more likely to practise in rural Manitoba.
Beginning in 2001, there have been comprehensive internal and external reviews of the Faculty's admission policies and procedures.The Brownell-Reiter External Review Report made a recommendation, among others, that policies and processes for medical school admissions be modified to increase enrolment of qualified students who would be more likely to practise in rural and remote areas in Manitoba 6 .
Increasing the proportion of medical students from rural areas has been recommended elsewhere as a focused way of addressing rural physician shortage 1,7,8 .
A 2002 study showed that students from rural Canada are underrepresented in medical schools in Canada compared with those whose parents are from urban Canada 9 .Only 10.8% of medical students have ever lived in a rural area compared with 22.4% of the Canadian population 9 .Such statistics suggest that people in rural regions may not have an equitable opportunity to gain entry into medical school.However, among medical school applicants, grades of rural-origin applicants and the proportion admitted were similar to those of applicants from urban backgrounds 10,11 .
Perhaps the issue is not that rural applicants are disadvantaged or discriminated by the admission process, but that a smaller-thanrepresentative proportion of prospective students from rural areas apply to medical school than their urban counterparts 12 .In fact, only 7.3% of Ontario applicants to medical school were of rural origin while 13% of the population of Ontario is rural 11 .Cultural and financial barriers likely play a bigger role in the decision to apply to medical school.
As in Ontario 10,11 , rural applicants from Manitoba, despite holding academic qualifications similar to applicants from urban Manitoba, were persistently under-represented in the medical school applicant pool and in the medical class, relative to the general population.Historically at the Manitoba medical school, rural classification, used merely to gather demographic data on Manitoba applicants, was based solely on location of high school graduation.In the years 2000 to 2007, while 28.5% of Manitobans lived in rural Manitoba, only 444 Manitoba applicants to medical school (21.8%) graduated from rural high schools in Manitoba.Similar to the proportion successful among urban applicants, 32.4% (n=144) of the rural applicants were successful in obtaining admission into medical school, and they represented 20% of the students in incoming classes in those years (Table 1).That is, the proportion of rural applicants offered admission showed no evidence of inequity compared with urban applicants, but the demographics of Manitoba medical school's registrants failed to represent the rural-urban demographics of Manitoba's population.In the absence of specific policies, rural applicants, underrepresented in the applicant pool, were also underrepresented in the incoming classes.Therefore, as a first step at the Faculty level, it was deemed critically appropriate to narrow the rural demographic gap in incoming classes by increasing offers of medical school admissions to qualified applicants from rural Manitoba.By integrating available evidence and physician workforce policy recommendations, the Faculty implemented new policies and procedures related to assessment of admissions criteria and ranking of applicants to medical school.The methodology adopted at this Faculty to bring about these changes may be of interest to educators and administrators responsible for rural health training and education in Canada and elsewhere.

The Working Group for personal assessment and scoring for admission to medical school
In April 2008, a Working Group was formed to develop criteria for personal assessment and scoring of applicants to increase the number of students admitted with a potential for rural practice.The specific objectives of the Group's mandate were: 1.To identify personal attributes reflecting a potential for living and working in rural areas and to determine a hierarchy of importance among these attributes expressed in their assigned relative weights.2. To develop a method to distinguish applicants with the personal attributes valued by the Working Group.

Methodology of the Working Group
Prior to the inaugural meeting, it was proposed that the methodology of the Working Group would include the following: 1. Literature review and environmental scan of practices used by Canadian and international medical schools to select applicants likely to serve in rural areas after completion of degrees.
2. Construction of a priority matrix to compare and rank personal attributes reflecting a potential for living and working in rural areas.
3. Design of a data-capture questionnaire focusing on personal attributes valued by the Working Group.4. Establishment of minimal criteria to be demonstrated by applicants receiving rural scores derived from the priority matrix.5. Selection of the magnitude of adjustment to applicants' composite scores via mathematical modeling of admissions data from past applicants.6.The outlining of procedures to be followed by the Admissions Committee while assessing and screening applicants for rural attributes, assigning rural scores, and applying adjustment constants before final ranking and selection of applicants.
The Working Group unanimously agreed to the proposed methodology (Fig1) at the first meeting.admitted; NR = total number of applicants graduated from rural high schools; nr = number admitted that graduated from rural high schools; NU = total number of applicants graduated from urban high schools; nu = number admitted that graduated from urban high schools.*Manitoba applicants include, where applicable, individuals who were in a special consideration category; †percentages across these 2 columns add up to only 90% because high school information on the 10% out-of-province applicants has not been provided here; ‡percentage of all 2041 Manitoba applicants in the years 2000-2007; §percentage admitted among applicants who graduated from rural high schools in Manitoba is not significantly different (p=0.75)from the percentage admitted among applicants from urban high schools in Manitoba.

Figure 1: Methodology followed by the working group for personal assessment and scoring of applicants with rural attributes for admission at the University of Manitoba Faculty of Medicine.
Literature review and environmental scan: Literature relevant to physician training for, and retention in, rural areas was circulated prior to the inaugural meeting.At the inaugural meeting, criteria [13][14][15][16][17][18][19] utilized by Canadian, American and Australian institutions to selectively admit applicants more likely to serve rural areas were discussed.

Rural practitioners and students interested in rural practice
were more likely to have rural connections such as rural upbringing and family in rural areas, to have graduated from rural high schools, to have lived in smaller communities, and to have rural service experience, irrespective of an urban or rural background [13][14][15][16][17][18][19] .

Construction of a priority matrix:
The priority matrix approach is used to plan for optimum results by providing a framework for setting priorities in complex situations with many factors 20 .It allows for the integration of available evidence with feelings, intuitions, and logic over multiple levels of decision making.The effective use of the priority matrix approach requires clarity of purpose and knowledge of options and choices that most contribute towards the successful accomplishment of that purpose.In this case, the purpose was to attribute relative weights to personal characteristics that reflect a potential for living and working in rural Manitoba.The personal attributes to be evaluated were chosen based on those identified in the literature (rural community service, rural connections, rural employment) as well as those honored historically at the University of Manitoba through our social accountability framework and commitments to educate the province's diverse communities (history of interprofessional work in health care, advanced degree, commitment to the Canadian Forces) (Table 2).
The matrix construct ( The chairperson ensured that the Working Group focused on the question at hand and that it did not change its thought process anytime during the exercise.At the completion of all possible pair-wise comparisons, an attribute score (ie a row total) was calculated for each attribute.The attributes with negligible or negative row totals were not considered further.Row total relative to the matrix total yielded the relative importance of the attribute as an indicator of potential for life and work in rural Manitoba compared with all other attributes in the context of the matrix.The relative importance of the attribute was also expressed as a percentage of the matrix total.
Personal Attributes Questionnaire to assess applicants' rural attributes: After establishing comprehensive definitions, a questionnaire to serve as a supplement to the main application form was prepared to enable the Admissions Office to elicit detailed, verifiable information on the attributes of interest.The questionnaire was constructed to encompass the different sections on the priority matrix: *Numeric values represented the following in pair-wise comparisons: 1/10 -Row attribute is much less important than column attribute 1/5 -Row attribute is somewhat less important than column attribute 1 -Row attribute is equally important as column attribute 5 -Row attribute is somewhat more important than column attribute 10 -Row attribute is much more important than column attribute.†Negative signs were introduced when attributes were thought to facilitate migration away from Manitoba rather than influence retention in Manitoba; ‡Attributes with negative or negligible row totals were not given further consideration beyond pair-wise comparisons; they are indicated here as darkened cells.
1. Rural connections, namely: rural birth, graduation from a rural high school, current primary address in a rural region, whether the first decade of life was spent in a rural area, whether the second decade of life was spent in a rural area.2. Employment in rural areas during high school, following high school, following an undergraduate degree.Only employment that lasted more than one semester or more than one summer was deemed as appropriate for consideration in this context.

Rural community service during high school,
following high school, following undergraduate degree.Service could be: (a) rural volunteer experience that lasted more than one semester or more than one summer; and/or (b) rural leadership experience that lasted more than one academic year or more than one season of sports.
'Rural' was defined, in the Manitoba context, as being outside the city of Winnipeg.The Faculty of Medicine reserved the right to verify all information reported on the supplementary questionnaire, and applicants were so advised.

Establishment of minimal criteria to be met during assessment of applicants' rural attributes and assignment of rural scores:
A defined minimum number of personal attributes within each section of the questionnaire was to be demonstrated before an applicant was assigned rural scores derived from the priority matrix.An applicant who reported at least 2 attributes under the rural connections section would be assigned a rural score of 40.0 (46.2%) of the total possible rural score of 86.6 from the matrix.One or more eligible rural employment experiences entitled an applicant to a rural score of 21.4 (or 24.7%).If any 2 eligible rural community service experiences were listed, then 25.2 (or 29.1%) would be the rural score assigned to the applicant.Applicants received scores for all the sections of the questionnaire for which they were qualified.For example, applicants qualified to collect scores on all 3 sections of the questionnaire would receive a rural score of 86.6 (or 100.0%).Applicants who do not meet any of the above criteria would receive a rural score of 0 (zero).factor 'p' will be no greater than 15%.That is, (15%*rural score) would represent a series of numerical constants whose values range from 0 for applicants with a rural score of 0, to 0.15 for applicants with a rural score of 86.6.In the end, the absolute increments to composite scores gained by applicants with rural attributes will be determined by applicants' rural scores as well as their composite scores.

Implementation of policy and procedures
The new policy and procedures were implemented during the application and admission cycle for the year 2009.In all, 129 of 327 applicants (39.4%) invited for personal interviews had rural scores in one or more of the  4).The change in the relative ranking of applicants after composite score adjustment is illustrated (Fig2).Forty-nine among the 60 accepted the offer of admissions and represented 44.5% of the 110 students in the incoming class.Thirty-three of the 110 incoming students (30%) graduated from a rural high school in Manitoba (Table 5), compared with the overall 20% of the incoming classes in the past who graduated from a rural high school in Manitoba (Table 1).This amounted to an approximate 50% increase in the representation of students who graduated from a rural high school this year, compared with previous years, and was closer to the proportion of Manitobans living in rural areas (28.5%).A short-term surrogate measure of success of the Working Group's work was that the rural demographic gap observed in previous incoming classes was met.
Table 3: Mathematical modeling of historic admissions' data from the University of Manitoba Faculty of Medicine: determining the percent increase to be applied to composite scores of competitive applicants with rural attributes so a sufficient number can be moved from waiting list to admit list

Applicants offered admission
Additional number of rural attributes applicants who would have been offered admission if their composite scores were selectively increased by:  *Mean score before adjustment of applicants with rural attributes did not differ significantly (P-value = 0.07) from applicants without rural attributes.†Mean score after adjustment of applicants with rural attributes did not differ significantly (P-value = .15)from applicants without rural attributes.‡Offers were closed when all 96 seats for Manitoba residents were accepted; at this point the adjusted composite score was 54.9.

Strengths of the priority matrix approach to enhancing diversity in medical class cohorts
By determining applicants' rural attributes and assigning corresponding rural scores, these revised admissions process identified and recruited 18 more students with rural attributes in the incoming class than would have been possible if relying on location of high school graduation alone (Table 5).Thus, the ability of the matrix approach to positively distinguish applicants as those characterized by rural attributes appears to be greater than the use of location of high school graduation alone.Moreover, the matrix approach does not exclude applicants from urban backgrounds who have demonstrated a rural orientation by their past life choices.It has been acknowledged that, in order to narrow the gap in rural physician shortage, a significant portion of physicians choosing rural practice will likely originate from urban backgrounds since students from urban backgrounds form the majority in medical schools 15,17 .
Yet another strength of the priority matrix approach was that it allowed for proportional adjustment of composite scores secondary to ordered increments in rural scores of applicants.

Generalizability of the Manitoba approach
Although the focus of this article has been specific to the Manitoba context, the approach can be easily applicable to all regional needs.The attributes evaluated in the priority matrix can be replaced or redefined, and made specific to any local context.The rural scores associated with attributes have the potential to change based on the number and type of other attributes in the matrix and also based on the Working Group evaluating the attributes.The minimal criteria to be demonstrated by applicants before rural scores can be assigned will also depend on local needs, values and communities of origin of applicants.For example, instead of stipulating that at least a decade of life must have been spent in a rural area in order to be eligible for rural scores under section 'rural connections,' other schools may wish to assign graded amounts of points for increasing numbers of years.
Finally, the priority matrix approach is highly adaptable and would be equally applicable to enhance diversity related to other attributes among eligible admission cohorts.To maintain transparency during admission process, the use of rurality scoring and the magnitude of adjustments made to the composite scores while ranking applicants should be communicated clearly to applicants, parents and other stakeholders.The procedures should preferably be described in the Faculty's applicant information bulletin.

Future directions
Having modified its recruitment and selection policies to moderately increase the proportion of students with rural attributes in incoming medical classes, the Manitoba Faculty is equally committed to monitoring and evaluating the effectiveness of this policy change, revising the policy as necessary, building the evidence base, and improving understanding about the extent to which admission interventions succeed in improving the health outcomes of rural populations.
An initial measure of the success of our intervention will be a sustained increase in individuals with rural attributes among enrolled students, followed by a demonstrable decrease in the shortage of rural physician workforce over time.Already a study is in place at the University of Manitoba medical school to track graduate outcomes such as residency choices and practice locations.In the future, outcomes will be related to rurality scores to evaluate effectiveness of the rural-cohort admission policy and to establish if selection of applicants by matrix methods ultimately results in greater levels of rural workforce.It is estimated that if at least half the additional 11 or so applicants with rural attributes who are offered admission annually due to the new policy choose rural practice in Manitoba, over the next 12 years there will be a minimum of 66 additional practitioners in rural Manitoba.This will be a 50% increase on the estimated number of Manitoba graduates entering rural practice in Manitoba currently.

Conclusion
Consistent under-representation of medical students from rural Manitoba during a period of rural physician shortage led the University of Manitoba Faculty of Medicine to use a qualitative, group decision-making tool to identify and rank attributes that were reported to select medical school applicants with a potential for living and working in rural areas.Using mathematical modeling on historic admissions data, the group recommended an adjustment factor that was hypothesized to narrow the rural gap in incoming classes.
The policy, at implementation, resulted in a class that was representative of the rural-urban demographics of Manitoba while maintaining the quality of selected applicants.As applicants with rural attributes were at least as academically qualified as applicants with no rural attributes, the authors agree with others 21 who suggested that medical schools can maintain competitive admission criteria while at the same time accepting students more likely to enter rural practice.

Figure 2 :
Figure 2: University of Manitoba Faculty of Medicine: distribution of composite scores and relative ranking of applicants considered for medical school admissions in 2009.
assessment and scoring for admission to medical school: Dr Samia Barakat, Dr Charles Bernstein, Dr Chris Burnett, Dr Dauna Crooks, Dr Lawrence Elliott, Ms Moni Fricke, Dr Don Klassen and Dr Daniel Sitar.

Table 2
on each column, and compared with every other attribute in a pair-wise manner (row-to-column comparison) to answer the question, 'which attribute, in the pair, is a more important indicator of potential for life and work in rural Manitoba?' Numerical values were entered to indicate whether the row attribute was considered by the Working Group to be more important than, less important than, or Negative signs were introduced when attributes were thought to facilitate migration away from Manitoba rather than influence retention within Manitoba.For each pair-wise comparison, values were entered simultaneously in the intersection of row with column, as well as column with row as the values were reciprocals of each other.

Table 2 : Construction of priority matrix to rank personal attributes reflecting a potential for life and work in rural Manitoba among applicants to medical school Which personal attribute is a more important indicator of potential for life and work in rural Manitoba? * † ‡
includes applicants with rural attributes who were not previously on the waiting list.Numbers in bold font indicate the number of applicants with rural attributes from the waiting list who would have been offered admission.