Original Research

Education to improve cancer care in rural South Australia

AUTHORS

name here
Elizabeth Hoon
1 PhD, Academic

name here
Jonathan Newbury
2 MD, Professor of Rural Health *

name here
Peter Chapman
3 MBBS, Chief medical advisor

name here
Joan Price
4 Senior project officer

CORRESPONDENCE

* Jonathan Newbury

AFFILIATIONS

1, 2 Spencer Gulf Rural Health School, Port Lincoln, South Australia, Australia

3, 4 Country Health SA, Adelaide, South Australia, Australia

PUBLISHED

21 April 2009 Volume 9 Issue 2

HISTORY

RECEIVED: 22 December 2008

REVISED: 6 March 2009

ACCEPTED: 21 April 2009

CITATION

Hoon E, Newbury J, Chapman P, Price J.  Education to improve cancer care in rural South Australia. Rural and Remote Health 2009; 9: 1147. https://doi.org/10.22605/RRH1147

AUTHOR CONTRIBUTIONSgo to url

© Elizabeth Hoon, Jonathan Newbury, Peter Chapman, Joan Price 2009 A licence to publish this material has been given to ARHEN, arhen.org.au


abstract:

Introduction: Cancer management follows the overall trend of rural health disparities, with higher incidence rates of preventable cancers and lower survival rates in rural Australia. Cancer prevention and management has been identified as a priority area and Cancer Australia has funded a variety of innovations throughout Australia. The Rural Chemotherapy Mentoring Program (RCMP) forms part of this drive to improve access to chemotherapy for rural based cancer sufferers in South Australia (SA). The key strategy of this program was the provision of opportunities for rural health clinicians (nurses and GPs) to enhance their knowledge and skills in the delivery of chemotherapy and cancer care through clinical placements at metropolitan oncology units. The RCMP enrolled 43 current SA rural clinicians (five GPs and 38 nurses). This evaluation was undertaken at the end of RCMP's initial 18 months. It considered how those involved in the RCMP perceived development and delivery of the RCMP, identifying key aspects of the program that were successful. This report emphasises lessons learnt which may be of relevance more widely in the development of other rural health professional education.
Methods: The evaluation used a mixed method approach, designed to accommodate different perspectives from the health professionals with different roles in the program. Quantitative and qualitative questionnaire data from clinician participants, their employers, and providers of education at metropolitan cancer units, were supplemented with qualitative interview data from these sample groups and from the program's steering committee. The analysis used interpretative methods to examine the key strengths, limitations and the potential for future development of the program.
Results: The vast majority of participants, employers, providers of training and the steering committee representatives expressed high levels of overall satisfaction about their involvement in RCMP. A clear identifiable need for increased knowledge and skills in cancer care was expressed by rural clinical participants pre-clinical placement. Technical aspects of the delivery of some chemotherapy protocols and the steps in safely preparing patients to receive chemotherapy were key areas where respondents lacked confidence prior to their clinical placement. Post-placement self-evaluation data highlighted improvements in participants' understandings and confidence about chemotherapy, and cancer care knowledge. Participants and their employers identified change in specific work activities resulting from their new knowledge. The program's limitations were identified.
Conclusions: Knowledge acquisition was a key to the success of the program, and the transfer of experience between rural and urban based clinicians was an important aspect of this. Generalisable recommendations for further improvement of the RCMP include: (1) clearly define and articulate a precise learning objective of the program; (2) involve staff who will directly deliver mentoring in planning of the program; (3) allow time in the planning phase to resolve complex indemnity issues across workplaces; and (4) provide funding for a dedicated trainer (preferably a staff member within the unit) to supervise placements in busy urban oncology clinics.

Key words: cancer care, education, rural clinicians.

full article:

Introduction

The management of health care in rural Australia is currently a major concern for rural communities, healthcare professionals, and government and non-government agencies. Official statistics continue to identify regional disparities in healthcare needs and health outcomes1,2. The National Health and Hospitals Reform Commission3 has recently identified improvements to Indigenous Australian's health status, and access to more equitable health services, as priorities for health care provision in Australia. Cancer management follows this overall trend of regional healthcare disparity. Despite some improvement, there remain higher incidence rates of preventable cancers4, and lower cancer survival rates for rural and regional residents5. Specifically, higher incidence rates of preventable cancers associated with excessive sun exposure (melanoma), higher smoking rates (lung, head and neck, and lip) and low pap-smear screening (cervical cancer) were identified for rural and remote Australians in 2001-20034. Cancer survival rates (for both 1 year and 5 year relative survival) also decreased with increasing remoteness (1997-2004) with lower survival rates particularly striking for rural and remote men5.

Federal and state governments have identified cancer prevention and management as a priority area for service improvement. Under the auspices of the National Health Priority Action Council (a joint initiative between federal, state and territory governments), cancer is one of five targeted areas of chronic disease management identified for coordinated action6. A National Service Improvement Framework for Cancer (2005)6 has been developed to inform policy-makers, health planners, consumer organizations health professionals and managers, about appropriate strategies for dealing with cancer prevention and management throughout the patient journey (National Health Priority Action Council). There has also been a focus on the development of research, programs, and professional development to support the cancer care workforce, under the coordination of the Federal government's Cancer Australia, established in 20067. The establishment of the Cancer Service Networks National Demonstration Program (CanNET)8 as a key program of Cancer Australia has also been developed to focus on rural service delivery and the development of better links between regional and metropolitan cancer services.

This Rural Cancer Mentoring Program (RCMP) initiative was funded by the Australian Government Department of Health and Ageing as part of the 'Strengthening Cancer Care Initiative' (http://www.health.gov.au/internet/budget/publishing.nsf/Content/health-budget2005-hbudget-hfact1.htm) prior to the establishment of CanNET. Country Health SA developed, co-ordinated and delivered the RCMP. Reflecting strategic priorities of improving cancer management, especially in regional Australia, the overall aim of the project has been to improve access and range of chemotherapy and cancer services to rural based cancer sufferers in South Australia (SA). In SA, rural and remote areas are sparsely populated with 85% of the state's population living in the state capital Adelaide. A wide range of health services are delivered by fly-in/fly-out day visits by medical specialists. The RCMP education program was developed so that rural cancer patients are able to receive more of their chemotherapy in their local rural health service. Further, the program aimed to enhance a functional connection between local rural health services and one or more of the urban specialist cancer services. These aims were operationalised through the provision of education for rural health clinicians (nurses and GPs) to enhance their knowledge and skills in the delivery of chemotherapy and cancer care education

An evaluation of the program was undertaken by the Centre for Rural Health and Community Development and the Spencer Gulf Rural Health School at the end of RCMP's initial 18 month funding period. This evaluation which focused on processes, impacts and outcomes of the education program, forms the basis of discussion in this article9. Specifically this article will consider how those involved in the RCMP (including participant clinicians, providers of education at metropolitan sites, employers and steering committee representatives) perceived development and delivery of the RCMP, identifying key aspects of the program which were successful. Emphasis will be given to lessons learnt for the future development of this program, and which may be of relevance more widely, in the development of other rural oriented education initiatives.

Methods

A mixed method approach was used in the collection and analysis of evaluative data (Table 1)10. Briefly, quantitative and qualitative questionnaire responses from clinician participants, their employers, and providers of the education at metropolitan hospital sites, were supplemented with qualitative interview data from these sample groups (and the program's steering committee). The analysis used interpretative methods to examine the key strengths, limitations and potential for future development of the program. An important component of the evaluation design was the accommodation of multiple perspectives from the varying groups involved in the program. While the views of people with cancer and their families were beyond the scope of this evaluation, the program's impacts on services and support for people with cancer are explored through qualitative data from participants and their employers.

Table 1: Evaluation instruments and responses by sample group



A series of questionnaires (pre-, post- and 6-weeks post-placement) developed and administered by Country Health SA provided quantitative and qualitative data about participants' understandings of chemotherapy and cancer care, and their experiences and reflections of their placement. The questionnaires predominantly used focussed 'closed' questions, although some 'open' questions were included to gain reflective data from participants. Likert scale questioning was used to ascertain participants' knowledge and skills, views of the program's processes, application of knowledge gained and professional support. As the pre- and post-questionnaires followed a similar structure it was possible to track change in attitudes and understandings at an aggregate level. A limitation of this anonymous questionnaire design was that it restricted participant profile data. Consequently, it was not possible to track change from pre- to post-placement for individual participants. Analysis of questionnaires (and interviews) and interpretation was formulated by the independent evaluators.

Quantitative analysis of the data used SPSS v15 (SPSS Inc; Chicago, IL, USA) to explore response patterns and change from pre- to post-placement attitudes. Given the number of cases, it was decided not to test for statistical significance tied to change or differences across variables or time. Rather, analysis focused on descriptive quantitative analysis. Analysis highlighted some concern about the way the evaluative questions/statements are presented in questionnaires. Despite piloting, the use of alternate positive and negatives statements invoked a possible double negative interpretation linked to complex questions about skill and knowledge. Data from these questions were consequently not used in analysis or interpretation.

Qualitative data were analysed to ascertain themes, patterns and differences and involved reading all transcripts, coding and summarising of trends. Following preliminary analysis of the Country Health SA data, the evaluation team conducted a series of semi-structured qualitative interviews (conducted face to face or by telephone) with a range of stakeholders to supplement the data already collected. These interviews allowed themes and questions raised from preliminary analysis to be pursued and the incorporation of perspectives from the steering committee which had not been canvassed. A key advantage of semi-structured interviews is that detailed questioning about issues raised within the interview may be pursued, along with emerging themes identified in concurrent analysis of the data. With agreement from respondents, most of the interviews were audiotaped for analysis purposes. The team continued to collect and analyse interview data from a range of participants, metropolitan site representatives, and steering committee until saturation of emergent themes was achieved. A summary of the data collection methods used in this evaluation is presented below.

Results

Participants


The RCMP operated between 2007 and June 2008. In all, 43 current SA rural clinicians (five GPs and 38 nurses) attended a five-day clinical placement at one of three cancer care specialist units within tertiary care hospitals in Adelaide, with the majority attending two of the centres.

Funding of the program covered participant salary costs and expenses tied to travel and accommodation, plus back-filling of their position while on placement. Salary funding was provided for a mentor at the metropolitan site for one day of the placement per participant.

It should be noted that the two primary host sites provided quite different placement opportunities and experiences. One of the cancer units provided participants with a structured timetable covering a variety of cancer care services, including radiotherapy, day units and ward experiences. Participants were assigned a mentor within the day unit. No practical experience was available at this site due to competencies and indemnity issues. Participants attending the other site spent 5 days in the cancer day unit and, rather than following a structured timetable, were absorbed into the work activities of the unit. Participants were not assigned a specific mentor, but the clinical nurse manager acted as primary contact for debriefing and monitoring of participant progress. One of the sites did not offer radiotherapy services. Towards the end of the placement at one site many participants undertook limited and supervised practical experience of changing peripheral intravenous central catheter (PICC) dressings.

A clear identifiable need for increased knowledge and skills in cancer care was expressed by participants' pre-clinical questionnaire. Technical aspects of the delivery of some chemotherapy protocols and the steps in safely preparing patients to receive chemotherapy were key areas where respondents lacked confidence prior to their clinical placement. Specifically, 80% of respondents reported a lack of confidence about the steps needed to prepare a patient for chemotherapy. Similarly, 87% of respondents expressed a lack of confidence about the technicalities of chemotherapy delivery (with almost half of the respondents moderately or strongly identifying with this statement).

Other areas of chemotherapy management and delivery where respondents reported a lack of knowledge and skill were:

  • assessment of clinical fitness of a person for chemotherapy: Almost half (49%) of the respondents lacked confidence.
  • discussing the purpose, effects and outcomes of chemotherapy with people receiving chemotherapy and their families: Over 60% agreed that they were not confident.
  • discussing long-term prognosis and/ or end-of-life issues with people receiving chemotherapy and their families: Almost 60% of respondents were not confident discussing this.
  • ability to respond to an incident with the equipment or materials during chemotherapy: In total, 62% of respondents were not confident in their abilities in this area.

While the majority of respondents felt that they understood the principles of chemotherapy delivery, one-third did not.

Overall, participants expressed greater confidence regarding issues associated with responding to the emotional needs of patients and their families (72% were confident); their own coping strategies for working with people with cancer (90% were confident); and communicating information about general health management to patients and families (87% were confident in this role).

Impact of the Rural Cancer Mentoring Program on knowledge of cancer care

While the response rate in this second phase of evaluation fell to 30 respondents (of 43 participants), aggregate data highlighted improvements in the general levels of knowledge reported by participants completing the five-day clinical placement. Many of the indicators used in this self-evaluation highlighted improvements in participants' understandings and confidence about chemotherapy, and cancer-care knowledge. When asked whether they understood the principles of chemotherapy delivery, all post- placement respondents agreed that they did. This is compared with one-third of respondents identifying that they did not, prior to attendance:



Figure 1: Change in levels of understanding of principles of chemotherapy delivery pre- and post-placement.


Positive qualitative comments highlighted improved understanding of specific knowledge bases, such as toxicities and, more generally, an increase in knowledge leading to greater confidence when caring for people with cancer:

Feel more confident in counselling patients prior to chemo and managing any complications after the placement.

This course was very helpful to me, to understand oncology and chemotherapy.


In the two key areas where participants reported a strong lack of confidence prior to attendance there were substantial improvements reported. Prior to clinical placements only 20% of respondents were confident about all the steps in preparing patients, compared with almost 80% post-placement. Similarly, only 13% of respondents expressed confidence relating to the technicalities of delivery prior to attendance, compared with 59% post-placement.

Post-placement all respondents reported that they were confident in their own coping strategies for working with people with cancer; monitoring patients during the delivery of chemotherapy and knowing the signs of possible complications; the removal and disposal of equipment; consulting palliative carers to discuss the needs of people with cancer; and consulting with other disciplines in their community who can assist in the care of families of people with cancer. In addition, all but one of the respondents were confident in the handling of hazardous materials, and keeping records up to date.

Flow-on effects tied to attendance on the Rural Cancer Mentoring Program

Participants and their employers identified change in specific work activities resulting from their program attendance. Many of these changes involved distribution of their acquired knowledge to other staff, or specific changes in procedures or practices associated with the care and safety of people with cancer:

Knowledge and information distribution: Several participants had or were planning in-service training sessions for local staff (one focussed on staff in the ward setting, and another on symptom management). Other participants reported involvement in the formulation of work instructions, policies and procedures, while several others reported more informal 'sharing of knowledge' and provision of support and counselling to nursing staff involved with cancer care. This included the circulation of information about support networks available (eg Cancer Australia). An employer also noted: '[Capacity increased] through providing up to date information to clients'.

Change in specific practices and/or procedures: Participants described how their role in chemotherapy care was developed by program attendance. One nurse has been able to relieve the chemotherapy nurse to administer 'chemo and other medications, eg intragam P, Remicade, blood transfusions etc., PICC line dressings, infusa port blood and Hepain locks'. Another nurse reported that s/he had independently administered chemotherapy; while a further participant has become a breast care nurse. There were also changes to specific practices associated with the administration and management of chemotherapy. Several nurses reported changes in their practices, including early intervention to avoid complications of extravasation and the taking of blood at least 2 days prior to treatment, along with improved patient observations (especially weight). Upgrading of equipment and antidote medications was also reported.

Improved service delivery to people with cancer: A participant reported how program attendance had resulted in the opportunity for people with cancer to access their treatments at home, rather than travelling to Adelaide. Another participant reported an improvement in patient care through a change in practices associated with prevention of extravasation at IV sites. A group of nurse participants were also planning to approach the local GPs with a view to improve communication about chemotherapy patients. Employers also reported change in practices:

More complex regimes of chemotherapy are now being undertaken.

Clients in our community having cytotoxic administration via Baxter or CHDD pump now; they have the choice of disconnection and flushing in the home. In one instance this saved a client travelling 100 km.

We hope to introduce insertion of PICC lines to our clients and this will support a wide range of clients within this region
.

Enhanced patient care and support: Many reported an increase in confidence and an improved effectiveness in addressing the needs of people with cancer and their families. One participant noted that they were better able to help people with cancer by being 'more caring and understanding both to the patient and the family'.

Limitations of the Rural Cancer Mentoring Program

The qualitative component of data collection provided insights into perceived limitations, from a variety of perspectives of those involved in the program. Three dominant themes were identified: (i) unmet expectations; (ii) difficulties tied to integrating education into a busy service situation; and (iii) quality and safety issues associated with expected prior knowledge of participants. These themes are related to, and to a certain extent reflect the differing perspectives of the stakeholders involved.

Unmet expectations: The lack of 'hands on experience' in the administration of chemotherapy, and limited opportunities for monitoring and managing patients through the chemotherapy process were raised by both participants and their rural employers. These comments reflected an expectation that the program would provide an opportunity to have clinical practice, but for many this had not been provided. Comments about lack of practice acknowledged the work demands of the staff at their placement sites, for example: 'I was not able to check chemos or put them up and that is what I thought I was going down for'.

There was also concern raised about the availability of mentors: 'I only had access to my mentor on the 4th day of the 5 day placement'.

The steering committee also acknowledged the lack of practice-based experiences in the placement, and the consequent lack of certification for participants as a limitation of the program in its current form. Representatives of the steering committee explained that CanNET is developing a strategy for cancer care service delineation, which will then be used to define competencies across cancer services in the state.

Difficulties tied to integrating education into a busy service situation: Both participants and staff at the host sites identified that the impact of the RCMP was felt primarily by the nursing staff in the cancer day units. Although funded for only one day of mentoring, nurses took on the role of teacher and mentor for the length of the placement. One respondent from a cancer unit reported that the mentor provided a 'mini chemo training session' for each participant. The impact of such an education program was understood to be accentuated because specialist cancer care units have multiple commitments to training programs that support rural as well as other clinicians. Host sites reported that they were also providing training to clinicians from the Northern Territory and were involved in other cancer care training initiatives.

Quality and Safety issues: Staff involved in the day-to-day delivery of the RCMP expressed concern that many clinician participants had limited knowledge of chemotherapy principles. Without basic knowledge there was concern that rural clinicians would gain little from the program. Staff at the host cancer units also reported that many of the RCMP rural clinicians had little chemotherapy experience. This meant that many of those charged with delivering the mentoring did not view participants as 'cancer safe'. The specialist chemotherapists saw rural clinicians as requiring more knowledge (to be safe) before acquiring skills. They also reported that a one-off 5 day placement does not allow acquisition of new chemotherapy skills. However they also believed that there was potential for their units to provide diverse experiences of cancer care, in radiotherapy and in ward and day units.

Broader health based contextual issues

Respondents (especially representatives of the steering committee) identified broader health based contextual issues that influenced the planning, delivery and impacts of the program. The role of medical oncologists was raised as highly influential in developing cancer care in rural communities, and consequently impacting on the impact of cancer care training initiatives. In particular it was recognised that:

  • Medical oncologists visiting rural centres provide a service that assists with patient travel and create/support the work of local, rural chemotherapy units.
  • Greater rural patient access (eg to prolonged initial consults) is dependent on adequate urban medical oncologists. There is currently a shortage of oncologists in the SA public health system
  • Small centres with occasional patients having chemotherapy are not favoured as the best sites for delivery of chemotherapy by many oncologists. It is anticipated that clinicians at such sites would require urgent re-training when new patients emerged.

Conclusion

The vast majority of participants, employers and providers of the placement at metropolitan host sites, and steering committee representatives expressed high levels of overall satisfaction about their involvement in RCMP. Knowledge acquisition was a key to the success of the program, and the transfer of experiences between rural and urban based clinicians was an important aspect of this enhancement of knowledge.

While rural clinicians successfully enhanced their knowledge about the principles, procedures and practices of cancer care, they gained limited practical experience within RCMP placement. Consequently their expectations of gaining skills experience were often unrealised. In contrast, specialist chemotherapy nurses, coordinators and managers at metropolitan sites viewed the RCMP placement as an opportunity for rural clinicians to gain cancer care knowledge. Without first acquiring appropriate and up-to-date knowledge, chemotherapists understood that rural clinicians should not be permitted to undertake 'hands on' practice within this program. Despite differences in clinical placements among the metropolitan sites, knowledge acquisition through observation was the key emphasis at both sites surveyed. This finding highlights the challenges of developing education that is appropriate not only for potential participants, but also fits within the safety and quality policies and procedures of host sites. This is of particular importance given the current focus on safety and quality in healthcare provision3. The finding also shows how learning objectives require clear articulation to all parties involved.

Because many rural clinicians had limited prior knowledge and/or experience in cancer care, and given the length and structure of this RCMP placement, it was not be possible to focus on up-skilling participants. However, now that the RCMP is established it may be possible to develop the program in ways that give participants ongoing up dates to cancer care knowledge, procedures and practices, including practice-based placements. The majority of the participants were enthusiastic about further education on cancer care. As the metropolitan sites offer different cancer care services, and operate under different procedures and protocols, their involvement in planning further program developments is crucial.

The workload pressures for nursing staff within the day units tied to delivering the RCMP placements were acknowledged by all sample groups. Staff took on substantial responsibilities in being a mentor, and covering for other unit staff while they undertook this role. The metropolitan sites reported difficulties in finding suitably qualified staff to backfill positions, reflecting the current skills shortage within many healthcare professions. In addition, funding was provided for one day of mentoring only while, in practice, mentoring and teaching occurred for the full length of the placement in the units. In developing an education program which utilises mentors, the level of responsibility and time involved in delivering this role requires careful consideration, given budgetary constraints. In the educational construct of 'knowledge, skills and attitude', the rural clinicians' attitude was of positively engaging in cancer work. They wanted to acquire additional skills but their urban colleagues saw their lack of knowledge as a limitation and were reluctant to teach them skills they would practise unsupervised in rural health services.

The influence of the broader healthcare context within which RCMP operated was also identified in this evaluation. The importance of metropolitan oncologists was highlighted, particularly in providing a regular point of contact for rural communities, and in assisting in the development of rural chemotherapy units and services. Without this contact people with cancer would be required to attend metropolitan sites for most treatments and assessment. Feedback from steering committee members highlighted the connections between shortages of medical oncologists in urban areas and services provided to rural patients. The current shortage of medical oncologists in metropolitan hospitals limits the time available for oncologists who do run rural clinics to be away from urban clinics. Consequently, initial consultations which take time and resources, are undertaken in the city. At the same time, participants and oncologists acknowledged that there are difficulties in keeping up to date with chemotherapy procedures and protocols when the demand for chemotherapy is occasional within small communities.

Recommendations pertinent for health based education program development include:

  • Clearly define the learning objective: there is a need to clarify the level of practical and observational experience offered on placement in order for participants to have realistic expectations. An understanding of the construct 'knowledge, skills and attitude' can guide this.
  • Providers of mentoring involvement in planning: it is important to fully involve those who will provide day-to-day mentoring, such as clinical nurse co-ordinators and managers at specific sites, in planning an education program.
  • Indemnity issues: resolving competency/indemnity issues in and among participating organisations is complex but worth pursuing in order to maximise the benefits achieved in clinical placements.
  • Provision of a dedicated trainer: it would be beneficial to fund a staff member at the host site to be dedicated to the role of training and mentoring participants throughout their placements.

Acknowledgement

The RCMP (SA) was funded by Australian Government Department of Health and Ageing as part of the 'Strengthening Cancer Care Initiative' (January 2007-June 2008).

References

1. Australian Institute of Health and Welfare (AIHW). Rural, regional and remote health: indicators of health status and determinants of health. Rural health series no. 9, cat. no. PHE 97. Canberra, ACT: AIHW, 2008.

2. Australian Institute of Health and Welfare (AIHW). Australia's Health 2008. Cat. no. AUS 99. Canberra, ACT: AIHW, 2008.

3. National Health and Hospitals Reform Commission (NHHRC). Beyond the blame game: accountability and performance benchmarks for the next Australian health care agreements. Canberra, ACT: NHHRC, 2008.

4. Australian Institute of Health and Welfare (AIHW) & Australasian Association of Cancer Registries (AACR). Cancer in Australia: an overview, 2006. Cancer series no. 37, cat. no. CAN 32. Canberra, ACT: AIHW & AACR, 2007.

5. Australian Institute of Health and Welfare (AIHW) & Australasian Association of Cancer Registries (AACR). Cancer survival and prevalence in Australia: cancers diagnosed from 1982-2004. Cancer series no. 42, cat. no. CAN 38. Canberra, ACT: AIHW & AACR, 2008.

6. National Health Priority Action Council (NHPAC). National service improvement framework for cancer. Canberra, ACT: Australian Government Department of Health and Ageing, 2006.

7. Australian Government. Cancer Australia. (Online) 2007. Available: http://www.canceraustralia.gov.au/ (Accessed 28 November 2008).

8. Australian Government, Cancer Australia. CanNET. (Online) 2007. Available: http://www.canceraustralia.gov.au/cannet-homepage.aspx. (Accessed 28 November 2008).

9. Hawe P, Degeling D, Hall J. Evaluating health promotion: a health worker's guide. Sydney, NSW: MacLennan & Petty, 1990.

10. Neuman W. Social Research methods: qualitative and quantitative approaches, 4th edn. Boston, MA: Allyn and Bacon, 2000.

You might also be interested in:

2021 - The renaissance of online medical education

2020 - Mechanisms to achieve a successful rural physiotherapy public–private partnership: a qualitative study

2014 - The right staffing mix for inpatient care in rural multi-purpose service health facilities

This PDF has been produced for your convenience. Always refer to the live site https://www.rrh.org.au/journal/article/1147 for the Version of Record.