What do evaluations tell us about implementing new models in rural and remote primary health care? Findings from a narrative analysis of seven service evaluations conducted by an Australian Centre of Research Excellence
Submitted: 23 February 2016
Revised: 10 April 2017
Accepted: 11 April 2017
Published: 6 September 2017
Citation: Lyle D, Saurman E, Kirby S, Jones D, Humphreys J, Wakerman J. What do evaluations tell us about implementing new models in rural and remote primary health care? Findings from a narrative analysis of seven service evaluations conducted by an Australian Centre of Research Excellence. Rural and Remote Health (Internet) 2017; 17: 3926. Available: http://www.rrh.org.au/articles/subviewnew.asp?ArticleID=3926 (Accessed 19 October 2017)
Introduction: A Centre of Research Excellence (CRE) in Rural and Remote Primary Healthcare was established in 2012 with the goal of providing evidence to inform policy development to increase equity of access to quality health care and the identification of services that should be available to the diverse communities characterising Australia. This article reports on the key findings from seven CRE service evaluations to better understand what made these primary health care (PHC) models work where they worked, and why. Methods: We conducted a narrative synthesis of 15 articles reporting on seven CRE service evaluations of different PHC models published between 2012 and 2015. Results: Three different contexts for PHC reform were evaluated: community, regional and clinic based. Themes identified were factors that enabled changes to PHC delivery, processes that supported services to improve access to PHC and requirements for service adaptation to promote sustainability. In both Indigenous and mainstream community settings, the active engagement with local communities, and their participation in, or leadership of, shared decision-making was reported across the three themes. In addition, local governance processes, informed by service activity and impact data, enabled these service changes to be sustained over time. The considerations were different for the outreach, regional and clinic services that relied on internal processes to drive change because they did not require the cooperation of multiple organisations to succeed. Conclusions: The review highlighted that shared decision-making, negotiation and consultation with communities is important and should be used to promote feasible strategies that improve access to community-based PHC services. There is a growing need for service evaluations to report on the feasibility, acceptability and fit of successful service models within context, in addition to reach and effectiveness in order to provide evidence for local dissemination, adaption and implementation strategies.
Key words: Australia, community engagement, narrative review, rural and remote primary health care.
Ready access to appropriate healthcare services is central to achieving optimal health outcomes. Nowhere is the challenge of improving access to health care more acute than in rural and remote communities. Geographic isolation, socioeconomic disadvantage, maldistribution of the health workforce and lack of resources are identified factors that contribute to the disparity in health outcomes experienced between rural and remote and metropolitan Australians1,2.
Since the early 1990s, there have been a number of federal and state policy and program responses in Australia, together with efforts by local health professionals and communities, to overcome these challenges1,3. In 2008, Wakerman et al4 reviewed rural and remote Australian primary healthcare (PHC) services, identifying features associated with successful PHC models, including adaptability to local contexts such as remoteness and population size, responsiveness to environmental barriers and enablers, and essential requirements for service sustainability. Despite this contribution, difficulties relating to access to services and service response to community needs persist and remain major impediments to improving the health and wellbeing of rural and remote populations.
A Centre of Research Excellence (CRE) in Rural and Remote Primary Healthcare was established in 2012 with the goal of providing evidence to inform policy development to increase equity of access to quality health care and to identify services that should be available to the diverse communities characterising Australia. The CRE sought to understand how healthcare services can be modelled to ensure their adaptability, acceptability and sustainability as well as the measures that policy-makers, service providers and communities should use to monitor the impact of improved access to appropriate PHC on health behaviour and health outcomes.
To achieve this, the CRE partnered with health service organisations and providers in three states and the Northern Territory to evaluate seven healthcare models that sought to improve the appropriateness, accessibility and acceptability of aged care, mental health services, comprehensive PHC and diabetes care in rural and remote communities. The heterogeneity of the studies, both in context and approach, is reflective of the fact that each evaluation was responsive to local needs and concerns. This article reports on the key findings from seven CRE service evaluations to better understand what made these models work where they worked, and why, taking into consideration features of implementation science5,6.
We conducted a narrative synthesis of 15 articles7-21, reporting on seven CRE service evaluations published between 2012 and 2015 (1–5 papers per evaluation). A narrative synthesis links valuable studies together for ‘reinterpretation or interconnection’22. Each evaluation addressed specific objectives developed with the responsible service agencies, governing bodies and/or practitioners. A summary of each service, the evaluation methods, and key findings is presented in Table 1.
The analysis elucidated factors from each evaluation that were associated with the development of the PHC service model, improved access to and sustainability of those services. Initial data analysis was conducted by DL and independently reviewed by ES. DL and ES both verified the identification of themes.
Table 1: Centre of Research Excellence in Rural and Remote Primary Healthcare primary healthcare service evaluations – reporting the service model, evaluation methods and key findings within the context structure
Three different contexts for reform were present between the seven CRE service evaluations:
Community-based reform: Four of the seven evaluations reported on community-based services: two innovative comprehensive PHC services - one in a remote Western Australian Indigenous community9-11 and the other in a small rural community in Victoria7,8, and one aged care service and one social and emotional wellbeing service – in remote Indigenous communities from the Northern Territory and Kimberley region of Western Australia respectively12,13.
Regional service reform: Two regional outreach service models were evaluated: provision of diabetes cycle of care to patients through the Royal Flying Doctor Service (South Eastern Section) network of fly-in fly-out clinics in far west New South Wales (NSW)14,15, and a virtual outreach, telehealth service established by a local health district (LHD) to improve access to emergency mental healthcare across western NSW16-20.
Clinic-based reform: The final service evaluation reported on modifications to a clinic process that introduced patient-led scheduling of appointments for routine mental health practice in a remote township of the Northern Territory to enhance the efficient and effective use of limited healthcare resources21.
Six of the seven new services were responsible for healthcare delivery in remote communities.
Three themes were identified from the evaluation reports: enabling changes to PHC delivery that resulted in more appropriate PHC services, processes that support services to improve access to PHC, and requirements for service adaptation to promote sustainability in changing internal and external environments.
Enabling change: Two of the community-based settings used a comprehensive PHC service to reorient existing services and extend service capacity to provide services that were previously not available to residents. Community ownership and leadership were reported as key drivers for significant changes to PHC service provision in both settings. The researchers reported that community capacity to articulate their health needs and negotiate with existing health services and government agencies over several years were critical elements that enabled change to occur.
To support this change, communities secured funding and agreement from key health providers and organisations in order to integrate their services and staff rather than duplicate efforts and to deliver services that better met community needs. Significant community consultation and engagement were critical early in the process when the proposal for change in community-based services was initiated by practitioners. Other key elements included the establishment of governance and operational processes to manage the complexity of multiple organisations being involved and to deal with expectations and funding issues, as well as potential external factors such as organizational priorities, reporting and accountabilities.
The case for change in both outreach regional services was organisationally driven, whereas for the mental health clinic it was practitioner driven. While the changes to service delivery were primarily internal to one organisation, consultation and/or negotiation with both health professionals and patients was undertaken.
Improving access: The establishment of new rural and remote PHC services required either significant redesign of existing services and/or the addition of new service elements to address identified service gaps.
The voice of community was influential in community-based settings where a challenge to usual practice and the redefining of roles and responsibilities for each participating service were required to support a new service model designed to improve community access to needed PHC services. In this setting, formal governance processes and ongoing community participation were identified as mechanisms to effectively manage the newly established cross agency collaborations and to ensure the new service remained focused on providing accessible, high-quality PHC.
The outreach regional services and clinic service were principally governed by internal organisational processes and used feedback from the CRE evaluation to monitor and respond to the impact of the changes on the use of, and access to, those specific services.
Service adaptation and sustainability: Features associated with sustainability of service delivery were identified from the experiences of three community-based PHC service models and the regional telehealth service, which had each operated continuously for 6 years by the end of the evaluation period.
Shared decision-making with communities enabled changes to PHC service delivery and their effective implementation. In the remote Indigenous communities, strong local leadership, the employment of local staff and development of leadership skills for local community members were used to promote culturally sound practices and community engagement with the new services. The routine collection and use of data to monitor service performance and assess need was employed to ensure the ongoing provision of appropriate high-quality care and inform service adaptation. This level of adaptability was viewed as necessary to respond to the external and internal changes, such as workforce turnover and varied funding allocations, that could affect service sustainability.
This article reports on a synthesis of the key findings from seven PHC service evaluations in rural and remote Australia conducted by our CRE to better understand how context influences what made these models work, and why. All the PHC services were successfully implemented to provide access to health care that was previously not available to residents. Our analysis represented a shift in focus from delineating key barriers and enablers for effective implementation to consider how those factors interact in real-world settings to enable the dissemination and implementation of evidence-based PHC services and programs.
The capacity for rural and remote health services to improve access to PHC remains variable, particularly in communities that are smaller and more isolated, and where local services may not be well designed to meet community needs or lack the resources to manage the changes required. Issues such as workforce organisation and supply, funding, governance and leadership, and infrastructure are associated with improved access to needs-based primary health care4, but how does the local context influence these factors? Greenhalgh et al. argued that achieving sustainable evidence-based changes in population health services needs to take account of the feasibility of proposed change at the local level (ie in context) and the acceptability and fit of those changes for practitioners, health agencies and patients/community, which is guided by ‘informed, shared decision-making with and by local communities’23.
We considered three contexts in this review: community, regional and clinic-based settings. While the questions in the evaluations were variously about key enablers of change and/or sustainability of the services, or accessibility, they nonetheless provided new insights into what was important in different settings to enable the introduction of changes that were feasible, acceptable and fit with the local context.
In both Indigenous and mainstream community settings the active engagement with local communities, and their participation in, or leadership of, shared decision-making with clinicians and health service organisations was reported as a key factor enabling those changes to occur. This included the complex task of getting different organisations to collaborate and integrate service roles in the community. Having initiated change, sustaining improved access continued to be guided by shared decision-making with community, linked to local governance processes and informed by service activity and impact data. The capacity for organisations to engage in this process, collaborate and integrate services at the local level appears to be a key feature that enables community-based PHC services in rural and remote communities to respond effectively over time to changes in the internal and external environment that impact on sustainability. Sufficient time to negotiate change is critical. Feasibility, acceptability and fit at the local level reflected a negotiated progress, not just an assessment.
The considerations were different for the outreach, regional and clinic services because the changes in these settings did not require the cooperation of multiple organisations to succeed. Furthermore, for the outreach services, the community of interest was more diffuse due to the dispersed nature of regional communities and patients who accessed those services. However, there were some similarities. The feasibility, acceptability and fit of the changes were informed by consulting with both health professionals who deliver or access the service, and their patients. The adaption of the service was informed by evaluation data that provided feedback about the use of, and access to, those specific services. These findings have implications for improving access to PHC in rural and remote communities that have the potential to reduce the health differentials between metropolitan and rural Australia.
Shared decision-making with communities can impact health outcomes and should be used more frequently to promote change in service delivery at the local level. Community engagement has a number of benefits, including the development of a shared responsibility between health services and community members to improve health outcomes, the efficient use of health resources, and a greater capacity to align healthcare delivery to community needs 1,3,24-27. This can be progressed by providing health services and practitioners with the skills and a mandate to effectively engage with their local community and, by investing in rural and remote communities, to develop their readiness and capacity to successfully work with health service agencies.
Chambers et al. argue ‘for understanding the changing context of healthcare to continuously refine and improve interventions as they are sustained’28. For too long, rural and remote health research has focused on describing program and service reach and effectiveness29. There is a need to conduct evaluation studies that examine and explain how key barriers and enablers for effective implementation interact in real-world settings. By examining the role and impact of context on the feasibility, acceptability and fit of those services an evidence base will develop to guide the dissemination of successful service models and the promotion of locally feasible implementation strategies.
The capacity for rural and remote health services to improve access to PHC remains variable despite a good understanding of the features associated with successful PHC models. The review highlighted that shared decision-making, negotiation and consultation with communities, tailored to context, is important and should be used to promote feasible strategies that improve access to PHC services. There is a growing need for service evaluations to include reporting on the feasibility, acceptability and fit of successful PHC service models within context to provide evidence for local dissemination, adaption and implementation strategies.
The review is a project of the CRE in Rural and Remote Primary Healthcare, Australian Primary Healthcare Research Institute. The information and opinions contained within do not necessarily reflect the views or policy of the Australian Primary Healthcare Research Institute, the Australian Government or the Department of Health. The CRE is a collaboration between the Broken Hill University Department of Rural Health, Centre for Remote Health–Alice Springs, and Monash University School of Rural Health–Bendigo. The Broken Hill University Department of Rural Health is funded by the Australian Government Department of Health.
1Standing Council on Health. National Strategic Framework for Rural and Remote Health. Canberra: Standing Council on Health, 2012.
2Health Workforce Australia. National Rural and Remote Health Workforce Innovation and Reform Strategy. Adelaide: Health Workforce Australia, 2013; 38.
3Standing Council on Health. National Primary Health Care Strategic Framework. Canberra: Department of Health and Ageing, 2013.
4Wakerman J, Humphreys JS, Wells R, Kuipers P, Entwistle P, Jones J. Primary health care delivery models in rural and remote Australia – a systematic review. BMC Health Services Research 2008; 8(1): 276.https://doi.org/10.1186/1472-6963-8-276
5Brownson RC, Colditz GA, Proctor EK (Eds). Dissemination and implementation research in health: translating science to practice. Oxford: Oxford University Press, 2012.
7Tham R, Buykx P, Kinsman L, Ward B, Humphreys JS, Asaid A, et al. Staff perceptions of primary healthcare service change: influences on staff satisfaction. Australian Health Review 2014; 38(5): 580-583.https://doi.org/10.1071/AH14015
8Buykx P, Humphreys JS, Tham R, Kinsman L, Wakerman J, Asaid A, et al. How do small rural primary health care services sustain themselves in a constantly changing health system environment?. BMC Health Services Research 2012; 12(1): 81.https://doi.org/10.1186/1472-6963-12-81
9Reeve C, Humphreys J, Wakerman J, Carter M, Carroll V, Reeve D. Strengthening primary health care: achieving health gains in a remote region of Australia. Medical Journal of Australia 2015; 202(9): 483-487.https://doi.org/10.5694/mja14.00894
10Reeve C, Humphreys J, Wakerman J, Carroll V, Carter M, O'Brien T, et al. Community participation in health service reform: the development of an innovative remote Aboriginal primary health-care service. Australian Journal of Primary Health 2015; 21(4): 409-416.https://doi.org/10.1071/PY14073
11Carroll V, Reeve CA, Humphreys JS, Wakerman J, Carter M. Re-orienting a remote acute care model towards a primary health care approach: key enablers. Rural and Remote health 2015; 15(3): 2942.www.rrh.org.au.
12Carey TA. A qualitative study of a social and emotional well-being service for a remote Indigenous Australian community: implications for access, effectiveness, and sustainability. BMC Health Services Research 2013; 13(1): 80.https://doi.org/10.1186/1472-6963-13-80
13LoGiudice D, Smith K, Shadforth G, Lindeman M, Carroll E, Atkinson D, et al. Lungurra Ngoora - a pilot model of care for aged and disabled in a remote Aboriginal community – can it work?. Rural and Remote Health 2012; 12: 2078.www.rrh.org.au.
14Kirby S, Moore M, McCarron T, Perkins D, Lyle D. Strengthening primary health care: achieving health gains in a remote region of Australia. Canadian Journal of Rural Medicine 2015; 20(2): 51-55.https://doi.org/10.5694/mja14.00894
15Kirby S, Moore M, McGarron T, Johnstone P, Perkins D, Lyle D. People living in remote communities can have best-practice diabetes care. Rural and Remote Health 2015; 15(3): 3203.
16Saurman E, Kirby SE, Lyle D. No longer 'flying blind': how access has changed emergency mental health care in rural and remote emergency departments, a qualitative study. BMC Health Services Research 2015; 15(1): 156.https://doi.org/10.1186/s12913-015-0839-7
17Saurman E, Johnston J, Hindman J, Kirby S, Lyle D. A transferable telepsychiatry model for improving access to emergency mental health care. Journal of Telemedicine and Telecare 2014; 20(7): 391-399.https://doi.org/10.1177/1357633X14552372
18Saurman E, Lyle D, Kirby S, Roberts R. Use of a mental health emergency care-rural access programme in emergency departments. Journal of Telemedicine and Telecare 2014; 20(6): 324-329.https://doi.org/10.1177/1357633X14544422
19Saurman E, Lyle D, Kirby S, Roberts R. Assessing program efficiency: a time and motion study of the Mental Health Emergency Care — Rural Access Program in NSW Australia. International Journal of Environmental Research and Public Health 2014; 11(8): 7678.https://doi.org/10.3390/ijerph110807678
20Saurman E, Lyle D, Perkins D, Roberts R. Successful provision of emergency mental health care to rural and remote New South Wales: an evaluation of the Mental Health Emergency Care-Rural Access Program. Australian Health Review 2014; 38(1): 58-64.https://doi.org/10.1071/AH13050
21Carey TA, Tai SJ, Stiles WB. Effective and efficient: Using patient-led appointment scheduling in routine mental health practice in remote Australia. Professional Psychology: Research and Practice 2013; 44(6): 405-414.https://doi.org/10.1037/a0035038
24Johnson A. Consumer and community engagement in health services: a literature review to support the development of an evidence based consumer and community engagement strategy for the Women's and Children's Health Network. Adelaide: Government of South Australia, 2015.
25Sarrami-Foroushani P, Travaglia J, Eikli M, Braithwaite J. Consumer and community engagement: a review of the literature. Sydney: University of NSW, Australian Institute of Health Innovation, Agency for Clinical Innovation, 2012.
26Fudge N, Wolfe C, McKevitt C. Assessing the promise of user involvement in health service development ethnographic study. British Medical Journal 2008; 336(313).
27World Health Organization. The World Health Report 2008. Primary health care – now more than ever. Geneva: WHO, 2008.
28Chambers DA, Glasgow RE, Stange CK. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation Science 2013; 8: 117.https://doi.org/10.1186/1748-5908-8-117
29Edwards N. Contexts, systems and networks; key considerations in joining up and integrating implementation research to improve population health outcomes. Canberra: Primary Health Care Research & Information Service, 2014.