Recent evidence shows over 20 million patients currently receive medical care at 1200 Community Health Centers (CHCs) in over 8000 site locations in the USA1,2. Community Health Centers provide affordable primary and preventative health care to at risk populations, including those on low-incomes, the uninsured, the unemployed, migrant workers, those who are homeless, racial and ethnic minorities, and other underserved populations2,3. The Public Health Service Act, Section 330, supplies funding for CHCs to deliver primary and preventative care to these medically underserved populations4. Recent initiatives have increased federal funding to introduce new CHCs and expand existing facilities in underserved areas5. This federal initiative to expand CHCs may exacerbate an already existing physician supply problem6,7, especially regarding those willing to locate to underserved areas8.
Another trend affecting the demand for physician services in underserved areas is the aging of Americans as baby boomers start to retire and require more medical attention. The United States Census Bureau predicted that the United States (US) population of age 65 years or older will grow by 60% between 2000 and 20309. In addition, the increasing attrition rate among aging physicians and population growth, particularly in the elderly population, will also affect physician supply10. Currently, there are approximately 7.3 million uninsured patients receiving care at CHCs, which increased from 3.9 million patients in 19982. The new health system reform law is anticipated to also contribute to the physician shortage due to the expansion of coverage to uninsured Americans and the subsequent increased demand at CHCs11,12. Rural areas contain 20% of the US population but only 9% of the US supply of physicians13,14. Almost half of CHCs are located in rural areas and these CHCs serve 10 million people or one in seven rural residents15. These rural areas continue to experience significant challenges in recruiting physicians16-18 despite creative medical school based initiatives focused on alleviating physician shortages in these underserved areas19,20. Similarly, initiatives in rural residency education have recently expanded and study of the outcomes for these graduate programs in producing physicians who practice in CHCs and other underserved rural environments is being reported21. Despite these supply side efforts, the production of physicians will not be able to increase at the same rate as the demand for their services. The cumulative impact of increasing demand for physicians as CHC numbers increase, demographic changes in the US population, physician attrition, the new healthcare reform law and the challenge of recruitment and retention of work force in rural areas will provide significant challenges to CHCs as they attempt to provide a qualified medical staff for their facilities.
The number of published studies that document successful case reports and/or strategies regarding CHC physician recruitment is limited. Barriers can often exist in the public sharing of information related to both successful physician recruiting and particularly as related to presentation or discussion of local challenges. As a result, many facilities and communities must rely on individual contractual assistance from physician recruitment firms and/or their own experience-based recruitment strategies without the benefit of a comparative, evidence-based or outside perspective. Without having an opportunity to identify and facilitate discussion of their communities' unique assets and capabilities and the ability to relate this information to regional phenomena, facilities and communities with historical difficulties in recruitment and retention of physicians may continue to experience physician shortage problems. Hesitancy to communicate even recognized shortfalls in order to prioritize efforts and effect cooperation between physicians and administrators or within the broader community can be an unintentional barrier.
The Critical Access Hospital Community Apgar Questionnaire (CAH CAQ) was developed to help rural communities address recruitment and retention challenges related to physician shortfall issues using a conceptual model that addressed the challenges identified22. The CAH CAQ terminology was adapted from the original newborn Apgar score developed by Virginia Apgar, which utilized five physiological classes (heart rate, respiratory effort, reflex irritability, muscle tone and color) to evaluate, and provide focused interventions for, an infant's medical condition twice in the minutes after birth23. Similarly, the CAH CAQ uses a community measurement scale comprised of five classes important to physician recruitment and retention (geographic, economic, scope of practice, medical support and hospital/community support) to provide a serial evaluation for intervention and planning twice across a given time span. Geographic class factors include spousal satisfaction, schools and the perception of the community. Economic class factors include loan repayment, signing bonus and part-time opportunities. Scope of practice factors include obstetrics, mental health and emergency room. Medical support factors include specialist availability, nursing workforce and call/practice coverage. Finally, hospital/community support factors include electronic medical records, welcome/recruitment program and televideo support. Across all of the classes, 50 factors were identified and each class contained 10 factors. The CAH CAQ is now being used in Idaho, Wyoming, North Dakota, Wisconsin and Alaska to assist critical access hospitals in physician recruiting efforts.
Based on this successful critical access hospital work, the Community Health Center Community Apgar Questionnaire (CHC CAQ) was developed. The purpose of this article is to report the findings from the validation research performed on the CHC CAQ during the development of this evaluation instrument. The CHC CAQ was developed to assess and differentially diagnose the strengths and challenges of an individual community as well as to provide information about the aggregate group. The CAQ assists in quantifying and visually demonstrating what has traditionally been a problem more commonly addressed by a more qualitative, expert-opinion approach. This instrument, accompanied by a growing database of aggregated data, also provides the opportunity for a community to gain peer comparison and time sequence comparison analysis. The results of this investigation may help CHCs in Idaho and other regions of the country to develop more effective recruitment strategies.
The CHC CAQ was developed based on the CAH CAQ. Researchers identified CHC-specific factors important in recruitment and retention through research, site visits to Idaho communities and discussions with physicians, CHC administrators and other professionals working to improve health care in underserved communities. As with the CAH CAQ, 50 factors were categorized into one of the following five classes: geographic, economic, scope of practice, medical support or facility/community support. Note that the hospital/community support class of the CAH CAQ was changed to facility/community support for the CHCs. Consistent with the CAH CAQ, each class of the CHC CAQ contained 10 factors. Further, 12 of the 50 (24%) CAH CAQ factors were different from those in the CHC CAQ to better represent physician recruitment and retention concerns of CHCs. These factors were housing, length of contract flexibility, perceived fiscal stability, retirement package, emergency/stabilization care, minor trauma, office gynecology procedures, pharmacy services, language services support, medical reference resources, delegated physician patient services and moonlighting opportunities. A series of three open-ended questions were also administered to validate the factors and identify any factor seen as significant but not addressed within the CHC CAQ. The CHC CAQ is provided in Appendix I and Appendix II provides a glossary of terms for the 50 factors in the CHC CAQ.
Selection and recruitment of target populations
The target communities for the CHC CAQ were all 13 CHCs in Idaho. One site was chosen per CHC if the CHC had multiple service locations. The site within each system with the largest medical staff and patient population served was selected as this site would be the major service access point for the CHC. The respondents were asked to answer the CHC CAQ based on an aggregate assessment across all locations if the CHC had multiple locations and their experience of the most active recruitment site. The degree of historical success in recruiting and retaining physicians in each community was identified by the researchers prior to the data analysis. Community Health Centers with more success in recruiting and retaining physicians were labeled as alpha or 'A' communities and those with less success were labeled as beta or 'B' communities. These assignments to either alpha or beta community status were based on statewide site visits, input from Idaho Primary Care Association employees and by experience in placing physicians in Idaho communities by physician leaders at the Family Medicine Residency of Idaho. The final sample included seven alpha (A) and four beta (B) CHCs for a total of 11 CHCs. Metro and non-metro or rural community classifications were identified using a 50 000 population threshold for rural communities24. There were seven out of 11 sites (63.6%) in the non-metro or rural category and four metropolitan communities (36.4%). Due to the rural nature of Idaho and the large underserved populations treated in a limited number of CHCs, even the most metropolitan-located Idaho CHCs frequently serve rural populations in satellite clinics. Two CHCs were excluded from the final sample, one due to potential conflict of interest issues (one principal investigator was employed by the excluded CHC) and another because the CHC employed only physician assistants rendering patient care.
The target population for the CHC CAQ was (i) the CHC administrator and (ii) physician leaders in these CHCs who had responsibilities for recruitment and retention activities. The physician leaders were selected in consultation with the CHC administrator. The recruitment of these individuals was done by phone and email by a family physician with physician recruiting experience and was supported by the Idaho Primary Care Association. There were 11 CHC administrators and 11 CHC physicians in the final sample for a total of 22 respondents.
Cross-sectional structured interviews were conducted in the study. The CHC administrators and physicians who agreed to participate in the study were mailed the CHC CAQ and a consent form after agreeing to participate in the study. A family physician with physician recruiting experience traveled to each participating community to administer the CHC CAQ. One-hour interviews were scheduled for each participant. Community Health Center administrators and physicians were interviewed separately and in private locations. Prior to the interviews, the consent form was reviewed with and signed by the participants. The CHC CAQ was completed during these structured interviews.
Data processing and analysis
The completed CHC CAQs were processed at Boise State University by researchers who entered these data into a Statistical Package for Social Sciences database. The qualitative questions were reviewed by the co-principal investigators and these responses are discussed in the Results section. Statistical Package for Social Sciences v17 (www.spss.com) was used for the statistical analysis. Descriptive statistics were used to organize respondent ratings to factors on the CHC CAQ. Numerical scores were constructed to describe sections in the CHC CAQ that address advantages and challenges, importance and Apgar scores. These score constructions are described more fully in the Results section. Descriptive statistics were employed to organize these results and Mann-Whitney U-tests were used for all tests of statistical significance reported in this research. These data have been stored in locked files and password protected hard drives at the Center for Health Policy at the College of Health Sciences, Boise State University and the Family Medicine Residency of Idaho. Access to the raw data has been limited to the principal investigators and qualified research staff.
This research was approved by the Boise State University Human Subjects Institutional Review Board (#EX 199-09-120).
As noted, 11 CHC administrators and 11 CHC physicians who had leadership roles in recruitment and retention participated in this study and completed a CHC CAQ in a structured interview format. The overall responses (n=22) for the CHC CAQ are provided (Table 1). The following sections describe the results for advantages and challenges ratings, importance ratings and Apgar scores by class.
Table 1: Distribution of responses across survey items
CHC CAQ advantages and challenges findings
Respondents were asked to rate their community's perceived advantages and challenges for the 50 factors in five classes. These factors were rated on a 4-point Likert scale (major advantage, minor advantage, minor challenge, major challenge) and these scale values were converted into scores (major advantage = 2, minor advantage = 1, minor challenge = -1, major challenge = -2). Statistical differences of these scores by class were determined between respondent and community types.
The advantages and challenges mean scores for the five classes within the CHC CAQ are provided (Table 2). Table 2 also contains p values for the statistical tests across occupation and community types. Class scores were calculated by summing scores across all 10 factors in a class. A summary score across classes was constructed by summing the class scores in the CHC CAQ. Medical support was identified as the highest community advantage followed by economic, geographic, facility/community support and scope of practice. There were no significant differences between CHC administrator and physician scores within or across classes. Comparisons between community types showed that A communities had significantly higher scores within four of the five classes (economic, p=0.04; geographic, p=0.05; facility/community support, p<0.001; and scope of practice, p=0.05) and across classes (p=0.001).
Table 2: Community advantages and challenges mean scores by class
Table 3: Community importance mean scores by class
CHC CAQ importance findings
Respondents were asked to rate their perceived importance levels for the 50 factors in five classes. These factors were rated on a 4-point Likert scale (very important, important, unimportant, very unimportant) and these scale values were converted into scores (very important = 4, important = 3, unimportant = 2, very unimportant = 1). Statistical difference of these scores by class was determined between respondent and community types.
The importance mean scores for the five classes within the CHC CAQ are provided (Table 3). Table 3 also contains p values for the statistical tests across occupation and community types. Class scores were calculated by summing scores across all 10 factors in a class. A summary score across classes was constructed by summing the scores across classes in the CHC CAQ. Geographic was identified as the highest area of importance for the communities followed by medical support, scope of practice, economic and facility/community support. There were no significant differences between CHC administrator and physician scores either within or across classes. Comparisons between community types showed that B communities had significantly higher scores for the geographic class (p=0.05). There was no across class significant difference in scores for A and B communities.
CHC CAQ Apgar findings
The following algorithm was used to calculate the Community Apgar score from advantage/challenge and importance scores: Community Apgar score = advantage/challenge score × importance score. The Community Apgar score ranges from -8 to 8 with a higher score indicating a more developed community asset and capability related to recruitment and retention of physicians. Statistical difference of these scores by class was determined between respondent and community types.
The mean Community Apgar scores for the five classes within the CHC CAQ are provided (Table 4). Table 4 also contains p values for the statistical tests across occupation and community types. Class scores were calculated by summing scores across all 10 factors in a class. A summary score across classes was constructed by summing the scores across classes in the CHC CAQ. Medical support was identified as the most significant community asset and capability followed by economic, geographic, facility/community support and scope of practice. There were no significant differences between CHC administrator and physician scores either within or across classes. Comparisons between community types showed that A communities had significantly higher scores within four of five classes (economic, p=0.01; geographic, p=0.04; facility/community support, p=0.001; and scope of practice, p=0.05) and across classes (p=0.001).
The cumulative Apgar scores for alpha and beta communities are provided (Table 5). Cumulative Apgar scores are a sum of the Apgar scores for each of the five classes in the instrument. The cumulative Community Apgar scores range from 389 to -44. Higher scores indicate greater community assets and capabilities. Generally, alpha communities have higher scores than beta communities.
Table 4: Community Apgar mean scores by class
Table 5: Cumulative community Apgar score by facility
The CHC CAQ contains three open-ended questions. Respondents were asked to identify the greatest barriers to recruitment and retention of family medicine physicians and potential solutions to overcome these barriers. They were also asked to identify the reasons why a successful physician candidate did not accept a position in the community and what that candidate ultimately did instead in terms of employment. The answers to these questions suggested that the CHC CAQ included all relevant variables related to recruitment and retention of physicians to rural communities.
Community health centers need tools to help them identify their communities' assets and capabilities related to physician recruitment, and that allow them to objectively identify strengths and opportunities for improvement. Previous workforce research on critical access hospitals produced an evaluation instrument (CAH CAQ) utilizing measurement classes (ie geographic characteristics, economic issues, medical support, scope of practice and hospital/community support) that have been found to be influential to physicians when deciding where to practice medicine. The purpose of this article is to report the findings from the validation research performed on the CHC CAQ during the development of this evaluation instrument. The CHC CAQ was developed to help CHCs assess and differentially diagnose the strengths and challenges of an individual community related to physician recruitment. This instrument also provides the opportunity for a community to gain peer comparison and time sequence comparison analysis. The results of this investigation may help CHCs in Idaho and other regions of the country to develop more effective recruitment strategies.
In these 11 Idaho communities, results regarding advantages and challenges identified medical support as the most advantageous class for physician recruitment followed by economic, geographic, facility/community support and scope of practice. For each class, there were no significant differences between CHC administrator and physician responses, demonstrating internal consistency in the identification of advantages and challenges for each factor. Significant differences in class and overall scores were seen between communities identified as alpha or beta for every class except medical support. Even then, alpha communities scored higher than beta communities for this class. These results suggest that the CHC CAQ consistently measures community assets and capabilities and correlates to historical experience in workforce trends for a particular community.
The results regarding importance highlighted geographic as the most important class for physician recruitment followed by medical support, scope of practice, economic and facility/community support. Comparison of community importance class scores between CHC administrator and physician responses identified no significant differences. Thus, all the respondents consistently recognized the classes important in recruitment and retention. On the other hand, one significant difference in importance levels of CAQ classes was observed between alpha and beta communities. Overall by category, beta communities gave greater importance scores to the geographic class. This is likely a result of the beta communities being more rural or isolated geographically. Generally, however, the overall importance scores demonstrated no practical difference between either the alpha and beta communities or between the administrator and physician respondents.
The overall rank ordering of classes by mean Community Apgar scores in these Idaho communities was as follows: medical support, economic, geographic, facility/community support and scope of practice. This may reflect the structure and financing of the CHC entities being better supported for their provision of a more limited scope of services as compared to critical access hospitals, while facing similar overall pressures for recruiting physicians as their critical access hospital facility counterparts. There are statistically significant differences within all classes and across classes with the exception of medical support, where alpha communities consistently scored higher on mean Community Apgar scores. Again, the presence of less of a difference between alpha and beta CHC communities across the medical support class may be in part due to the scope of services and their organization as CHCs. Caution should be exercised, however, given the limited sample size, which makes further investigation of this point necessary. Statistical differences were not found by respondent type within any class or across classes.
A review of the cumulative Community Apgar score by facility results suggest that the CHC CAQ consistently both quantifies self-report of community assets and capabilities and furthermore correlates to historical experience in workforce trends for a particular community. While 'Community 8' scored higher than two of its alpha counterparts, the trend clearly identifies a gradient effect between the higher scoring alpha communities and the lower scoring beta communities. The phenomenon of these findings can be explained by at least two observations. First, communities do not remain static within their historical categorization of alpha or beta but do in fact improve (or devolve) in their abilities and assets. Second, this study enrolled all eligible CHCs with a gradient from alpha to beta that was defined in relative terms to one another. The prior study of critical access hospitals referenced earlier in this report was a sample of 11 alpha and beta communities selected from a total of 26 critical access hospitals which allowed the researchers to identify facilities that were clearly more or less successful in physician recruitment. This selection provided a sharper distinction in Community Apgar Scores as communities that were neither clearly alpha or beta and were not included in the sample.
One of the limitations of this study was the small number of sample communities. The target communities for the CHC CAQ were all CHCs in Idaho and one site was chosen per CHC if the CHC had multiple service locations. Although a careful selection process was introduced to assure the quality of samples, they may have been biased and may not represent the target population. Another limitation of this study was the data collection method. Because the face-to-face interview method was used in the study, a response bias may have occurred. In addition, although the use of non-parametric tests was appropriate considering the sample size and the format of the instrument, some significant relationships may not have been detected due to the limited statistical power. Finally, the CHC CAQ may not have captured all relevant factors related to physician recruitment to CHCs. An examination of the qualitative questions in the CHC CAQ indicated that respondents did not identify any additional variables of interest.
Similar to the critical access hospital CAQ study, the CHC CAQ seems to not only discriminate between communities with greater assets and capabilities and those with lesser assets and capabilities but also to accurately correlate to historical community-specific workforce trends. This assessment may allow for identification of both modifiable and non-modifiable factors and also may suggest which factors are most important for a community to address with limited available resources. The CHC CAQ may also have a role to play in a community's self-evaluation, prioritization of improvement plans, advertising considerations and negotiation strategy for successful recruitment and retention of physicians.
Following the work already underway in critical access hospital communities, this tool may also be used to share successful strategies communities have implemented to overcome disadvantages that may be difficult or impossible to modify. The ongoing study of both community health center and critical access hospital community settings in the framework of the Community Apgar Questionnaire and the associated Community Apgar Program, focused on making improvements from the obtained information, will provide both cross-study data between these settings as well as ongoing temporal data for identification of longitudinal trends, aggregate analysis and targeted individual community benefit. The development of an aggregate CHC CAQ national database composed of multiple state data sets will allow for comparison and contrast of factors important to physician recruitment and retention both within and between states. For example, it may be useful to assess across states or within regions the differential impact of (i) the common finding of unmet mental health needs; (ii) how the quality of schools effect physician recruitment to rural communities; (iii) physician loan repayment, salary and production issues; and (iv) the changes regarding internet availability and electronic medical records. The results of such studies could inform regional and national leaders and policy makers as they craft legislative efforts, educational programming models or other approaches to addressing physician shortages to underserved communities.
This research was funded by the Idaho Department of Health and Welfare, State Office of Rural Health and Primary Care through a grant from the US Department of Health and Human Services, Health Resources and Services Administration, Office of Rural Health Policy, CFDA #93.913. Financial contributions to this research were also made by Boise State University and the Family Medicine Residency of Idaho. The authors thank Denise Chuckovich, Executive Director, and Katrina Hoff, Director of Workforce Development, of the Idaho Primary Care Association for their assistance in this research. The authors also thank Bradley Morris, Center for Health Policy, for his editorial assistance on this manuscript.
1. Adashi EY, Fine MD, Geiger HJ. Health care reform and primary care - The growing importance of the community health center. New England Journal of Medicine 2010; 362(22): 2047-2050.
2. National Association of Community Health Centers. America's Health Centers. (Online) 2011. Available: http://www.nachc.com/client/documents/America's%20Health%20Centers%20updated%20August%202011.pdf (Accessed 21 February 2012).
3. Hawkins D, Groves D. The future role of community health centers in a changing health care landscape. Journal of Ambulatory Care Management 2011; 34(1): 90-99.
4. Frick KD, Regan J. Whether and where community health center users obtain screening services. Journal of Health Care for the Poor and Underserved 2001; 12(4): 429-445.
5. Wells R, Morrissey JP, Lee IH, Radford A. Trends in behavioral health care service provision by community health centers, 1998-2007. Psychiatric Services 2010; 61(8): 759-764.
6. American Academy of Family Physicians. Family physician workforce reform: Recommendations of the American Academy of Family Physicians, Draft Reprint No. 305b. (Online) 2006. Available: http://www.aafp.org/online/en/home/policy/policies/w/workforce.html (Accessed 14 February 2012).
7. Center for Workforce Studies, Association of American Medical Colleges. Recent studies and reports on physician shortages in the U.S. Washington, DC: Association of American Medical Colleges, 2009.
8. Kuehn BM. Reports warn of primary care shortages. JAMA 2008; 300(16): 1872-1875.
9. United States Census Bureau. U.S. interim projections by age, sex, race, and Hispanic origin: 2000-2050. (Online) 2004. Available: http://www.census.gov/population/projections/data/national/usinterimproj.html (Accessed 26 November 2012).
10. Hancock C, Steinbach A, Auerswald CL, Nesbitt TS, Adler SR. Why doctors choose small towns: a developmental model of rural physician recruitment and retention. Social Science and Medicine 2009; 69(9): 1368-1376.
11. Hofer AN, Abraham JM, Moscovice I. Expansion of coverage under the Patient Protection and Affordable Care Act and primary care utilization. Milbank Quarterly 2011; 89(1): 69-89.
12. Mann S. Addressing the physician shortage under reform. Association of American Medical Colleges. (Online) 2011. Available: https://www.aamc.org/newsroom/reporter/april11/184178/addressing_the_physician_shortage_under_reform.html (Accessed 26 November 2012).
13. Council on Graduate Medical Education. Tenth report: physician distribution and health care challenges in rural and inner city areas. (Online) 1998. Available: http://www.hrsa.gov/advisorycommittees/bhpradvisory/cogme/Reports/tenthreport.pdf (Accessed 26 November 2012).
14. Eberhardt MS, Ingram DD, Makuc DM et al. Urban and rural health chartbook. Health, United States, 2001. Hyattsville, MD: National Center for Health Statistics, 2001.
15. National Association of Community Health Centers. Removing barriers to care: community health centers in rural areas. (Online) 2011. Available: http://www.nachc.com/client/documents/Rural%20Fact%20Sheet%20November%202011.pdf (Accessed 8 March 2012).
16. Pepper CM, Gray MJ, Sandefer RH. Recruiting and retaining physicians in very rural areas. Journal of Rural Health 2010; 26(2): 196-200.
17. Rosenblatt RA. A view from the periphery - health care in rural America. New England Journal of Medicine 2004; 351(11): 1049-1051.
18. Rosenblatt RA, Andrilla CH, Curtin T, Hart LG. Shortages of medical personnel at community health centers: implications for planned expansion. JAMA 2006; 29(9): 1042-1049.
19. Quinn KJ, Stevermer JJ, Kane KY, Williamson HA, Hosokawa MC, Webb WD et al. Influencing residency choice and practice location through a longitudinal rural pipeline program. Academic Medicine 2011; 86(11): 1397-1406.
20. Rabinowitz HK, Diamond JJ, Markham FW, Santana AJ. Increasing the supply of rural family physicians: recent outcomes from Jefferson medical college's physician shortage area program (PSAP). Academic Medicine 2011; 86(2): 264-269.
21. Patterson DG, Longenecker R, Schmitz D, Xierali IM, Phillips RL, Skillman SM et al. Rural residency training for family medicine physicians: graduate early-career outcomes. Policy brief. Seattle, WA: WWAMI Rural Health Research Center, University of Washington, 2012
22. Schmitz DF, Baker E, Nukui A, Epperly T. Idaho Rural Family Physician Workforce Study: the Community Apgar Questionnaire. Rural and Remote Health 11: 1769. (Online) 2011. Available: www.rrh.org.au (Accessed 25 November 2012).
23. Apgar V, Holaday DA, James LS, Weisbrot IM, Berrien C. Evaluation of the newborn infant -second report. JAMA 1958; 168(15): 1985-1988.
24. Office of Management and Budget. 2010 Standards for delineating metropolitan and micropolitan statistical areas; notice. (Online) 2010. Available: http://www.whitehouse.gov/sites/default/files/omb/assets/fedreg_2010/06282010_metro_standards-Complete.pdf (Accessed 25 November 2012).
Appendix I: Community Health Center Community Apgar Questionnaire
Appendix II: Community Health Center Community Apgar Questionnaire glossary of terms