A qualitative evaluation of remote supervision guidelines for Australian general practice registrars in two practice locations

Introduction

There is an urgent and increasing need to build and sustain the Australian rural and remote general practice workforce. Approximately 29% of Australians live in rural and remote areas, where the ageing population has a higher mortality and morbidity rate due to chronic disease, multimorbidity and injury¹.

Many rural and remote doctors have little contextualised experience, appropriate qualifications or support for the work they are doing². GPs may be overloaded with their own clinical practice and not have the time or enthusiasm to supervise a general practice trainee (registrar). Training registrars in rural and remote settings is essential to develop GPs with relevant skills for the scope of practice and context required³.

After many years of remote supervision in Australia, there is no evidence of harm as compared to onsite supervision⁴. Evidence shows that a well-supported model designed specifically for remote supervision ‘facilitates the creation and maintenance of professional connections and support’⁵.

Good supervision relies on a strong educational alliance and requires a ‘sound professional relationship, self and mutual awareness of strengths and weakness, confidence to seek and provide help at any time, and the ability to provide and receive appropriate feedback’⁶ so that the registrar is safe to ‘reveal and address weaknesses in his or her knowledge, skills and emotional responses to practice’². It is more than clinical advice, but includes attending to the registrar’s wellbeing, supporting them through challenging situations, and brokering their relationship with the practice and the community.

In effective remote supervision, the registrar is also supervised by a team, including Aboriginal Health Workers/Practitioners, practice managers, nurses, tertiary services, locums and other multidisciplinary, community-based health service providers. This not only expands clinical knowledge but builds an ‘amalgam of role models and richer learning than interaction with a single supervisor’⁴.

Remotely supervised registrars will promptly find evidence-based and contextually appropriate answers to problems by contacting the remote supervisor, other members of the supervision team, or referring to written guidelines. Patients often prefer seeing a registrar to a long wait for a visiting specialist or travelling away to a distant hospital⁴. ‘There is a perceived increase in the quality and continuity of healthcare resulting from more trainees working in the community, rather than an ever-changing supply of locums … Therefore, there is improvement, not only in overall health status and outcomes, but also specifically to accessibility, continuity and quality of healthcare, including preventive healthcare services, and the availability and sustainability of the rural general practice workforce’⁷.

A balance needs to be reached between the level of support versus the challenges a registrar confronts. For instance, those who have supervisors nearby may ask for assistance without going through the full process of problem-solving and clinical reasoning they would use if supervised remotely^2,4,6. Remotely supervised registrars are likely to be more resourceful in seeking answers to clinical questions⁸, receive feedback on clinical decisions they make autonomously^2,9, learn how to cope when immediate advice is unavailable, and develop skills to assist them with practising independently¹⁰.

Rather than viewing remote supervision as ‘second-best training’, guidelines have been developed by the Royal Australian College of General Practitioners (RACGP) to facilitate supervision and learning that is comparable to, if not better than, traditional supervision.

The Australian Department of Health and Aged Care funded two projects to develop, pilot, evaluate and refine the remote supervision guidelines (Remote supervision: Guidelines for safe and effective general practice training utilising remote supervision) in 2021–2023.

Extensive research and consultation with stakeholders informed the guidelines’ development, including an international literature review and environmental scan of practice in Australian general practice training organisations, 50 semi-structured interviews with stakeholders and an expert advisory group.

There are several key differences between the established face-to-face supervision, and remote supervision, as outlined in the guidelines:

• selection of supervisor and registrar using a contextualised remote supervision placement process (CRSPP)
• remote supervision risk management plan
• face-to-face-orientation period
• development of an onsite supervision team
• communication strategies for clinical, professional and personal support, and assessment using it
• acknowledgement of the increased time required for remote supervision with additional payment for offsite teaching, support and assessments.

In October 2021, the RACGP remote supervision pilot was conceived to evaluate the guidelines, processes and documentation in the pilot sites. This research aimed to assess the practicality, safety, effectiveness and efficiency of the guidelines for the remote supervision of registrars in the pilot locations.

Methods

In 2022 the guidelines were piloted in Walgett Aboriginal Medical Service and Norfolk Island. Walgett is a town of about 2000 people in rural northern New South Wales. Aboriginal people make up 43% of the population. There is an Aboriginal Medical Service, a mainstream medical centre and a ‘multipurpose service’ with an emergency department, inpatient and residential aged care beds. The nearest large centre is Dubbo, which is 186 km away. Norfolk Island is in the Pacific Ocean, equidistant between New Zealand and New Caledonia, 1412 km east of Australia, with a local population of about 2000 people and often an equal number of tourists. Norfolk Island Health and Residential Aged Care Service provides general practice services, emergency services, inpatient and aged care facilities, and community health. Visiting specialists fly in from Brisbane.

The pilot aimed to explore whether remote supervision placements were safe, supportive, and provided a quality training experience; to establish whether the training requirements and risks were assessed appropriately; and to review how the guidelines, processes and documentation could be improved.

The pilot also explored whether remotely supervised registrars were adequately mentored and supported clinically and professionally by their remote supervisors and could demonstrate the ability to work successfully with their onsite teams. The aim was to review and refine the guidelines prior to publication and wider implementation.

The focus of this evaluation was on the experience of the supervisors, registrars and practice site in using the guidelines and whether they perceived that the guidelines enabled a safe placement with quality training. A single-case phenomenological approach was therefore taken. The two medical educators who developed the guidelines have extensive experience in rural and remote locations whereas the other authors come from different backgrounds and have different skills and expertise.

The interviews with the supervisors and registrars were conducted by the medical educators who developed the guidelines, with an emphasis that the aim was to improve the guidelines for the future. Any potential conflicts were discussed, but none of the participants felt that this influenced their contributions.

Several activities were incorporated into the evaluation:

• desktop review of key relevant documents
• semi-structured interviews (conducted by the medical educators) with the remotely supervised registrars (two), the supervisors (three), and representatives from the practices (two) at three time points: before the placement began, after the 2-week orientation period and towards the end of the placement
• interviews with six key placement stakeholders to identify perceived strengths and weaknesses in the placements. These were conducted by the project manager.

The questions were developed by the project team and were based on the guidelines. They sought to understand the experience of the participants and were constructed to align with the placement activities at differing time points (Table 1).

All interviews were conducted using Zoom and were transcribed by an independent transcription service. Interviews ranged from 30 to 60 minutes. Codes were generated from the first round of interviews using NVivo v20 for Windows (Lumivero; https://lumivero.com/products/nvivo) by one researcher and using a manual process by the other based on Braun and Clarke’s topic approach to thematic analysis¹¹. The two analysts undertook an observational review of select coding and analysis for the purpose of inter-rater reliability checking. No significant areas of disagreement between them were identified. They then developed a final coding structure based on the research questions, which was used to analyse the rest of the interviews. Further codes were discussed as they developed.

Ethics approval

This research gained ethics approval from Flinders University Research Ethics and Biosafety Committee, project number 5532.

Results

Overall, the participants and stakeholders in the pilot were very positive. Both registrars continued to work in the practice in which they trained remotely after they completed their training.

The qualitative results are divided according to three time points: pre-placement, post-orientation and a final interview after the completion of the placement.

Pre-preplacement period

Registrars and supervisors indicated that the CRSPP enabled them to understand and fill current knowledge gaps, and identify what supports were available; it was an effective tool to match the registrar, supervisor and location.

Actually, there were some clues in it but I felt it sort of more gave us a feel for her as a person and I suppose the whole thing about the remote supervision is the fit of the supervisors and the registrar. (Supervisor)

Before the placement started, risks were assessed under the headings of ‘Personal’, ‘Registrar–supervisor relationship’, ‘Training’ and ‘Environmental’ using the risk assessment and management matrix.

I thought it was straightforward a good list and I thought it was helpful in terms of me understanding what the differences would be in between a normal supervisor on site and this type of remote supervision. (Accreditation officer)

Suggestions for improvement for activities in the pre-placement period

It is important to ensure clear expectations are set regarding aims and outcomes of the CRSSP. One registrar thought the CRSSP process would be akin to a formal job interview.

One remote supervisor suggested to ‘blind’ the CRSSP scenarios to prompt a more ‘spontaneous’ and ‘valid’ result.

Challenges during the pre-placement process

Communication between stakeholders and role clarification was an issue for some of those outside the remote supervision team.

… it has required quite a bit of time involvement for everyone involved, not just myself, but also [RACGP administrator], the accreditation team, the medical educators in the regional teams being involved with the interviews. (Deputy director of training)

There were also issues between IT platforms and integration of systems across training sites.

It was noted that the forms and documentation would need to be reviewed to ensure they are appropriate for usual training needs (as opposed to the pilot program), with ‘packages’ for the various stakeholders (eg practice, medical educator, training coordinator); the need for clear documentation of the timelines and sequence of processes; and transparent financial arrangements.

Post-orientation period

The purpose of the supernumerary capacity during orientation was misunderstood by some, while others thought it assisted in building a strong relationship between supervisor and registrar.

… it was encouraging having that conversation with you prior to the orientation when you kind of reiterated that it was about building a relationship. (Supervisor)

I probably would’ve quarantined some more one-on-one in the early days, rather than that mixed model of care that we had done. And probably would have made sure that [the registrar] wasn't on call for those first two weeks while [the registrar] found [their] feet … (Training site)

Overall, interviewees reported that the orientation period provided a safe and supportive training experience, and enabled expectations to be set.

My orientation was really good. I [came out of it] feeling really supported and I feel like [supervisor] in the first week have set very clear boundaries with the practice that I was not to be used as another full-time GP and that I’m still learning and that teaching sort of learning space, breathing space needs to be protected. (Registrar)

Supervisors and registrars expressed confidence in the initial registrar assessments and the identified call-for-help list.

An effective onsite supervisory team was established in both locations.

Suggestions for improvement from post-orientation interviews

Aboriginal Medical Services may need a different approach, especially with the development of the onsite team. The importance of cultural mentoring was highlighted and the difficulty finding someone appropriate.

… there are some really expert people in AMS [Aboriginal Medical Services] teams as you well know, who achieved a lot of things that the GP or the registrar can't do and aren't expected to do. (Supervisor)

I do believe some of that local knowledge and information, it got lost a little bit because the focus became on doing the doctoring stuff and being a GP. (Training site)

… I still feel like a fish out of water culturally. I’m learning little bits here and there through my patients … But I don’t really have a strong connection to the culture or to the land and I think that would make a big difference in my job satisfaction here if I did. (Registrar)

Challenges during the placement

The timelines for the development of a remote supervision placement were not transparent to all the stakeholders. The roles, communication and purpose of the orientation period should be outlined well in advance.

Documentation for the orientation period needs to be presented more clearly and effectively as a single simplified document, as there was some confusion around what was required.

It felt like nobody kind of knew whose job it was to do which, and so everyone was doing everything. (RACGP administrator)

Final interviews

Interviewees reported that the weekly remote supervision activities were not always completed due to environmental and resourcing factors. However, registrars and supervisors did feel supported by these activities.

These activities met the needs of the remotely supervised registrar and remote supervisor, and the approach outlined in the guidelines around having the ‘right registrar and the right supervisor’ was achieved.

[Supervisor] was my day-to-day contact, which was awesome. [Supervisor] and I had some great chats on the fly, just mostly about patients. Sort of towards the end, there was a little bit of pastoral care and more of emotional deeper relationship that developed which was really nice. (Registrar)

… it’s almost like registrars may have to be handpicked for the right location and a supervisor that’s the right supervisor for them. (Supervisor)

There was a comprehensive risk management process throughout the placement at one of the training sites. However the other training site reported that they were unaware of the risk matrix.

They wanted to make sure that they were doing everything possible from a training organisation to see what the risks are at all levels. Whilst for us here at our end, it's been a living document, and we've reviewed each thing as we're going. (Practice manager)

Most of the stakeholders understood the process of remote supervision, and believed the documents were practical, useable and clear.

So, the risk matrix, I think that’s great … if a registrar didn’t go through those risk management matrixes and something happened, and then at the end of it, they went, ‘Well, I wasn’t told,’ that’s a problem and I definitely was told all of that and I understood it. (Registrar)

Importantly, remote supervision placements were attractive to registrars, supervisors and practices. They were viewed as providing quality training experiences and were safe for the registrar, supervisor and community.

I think this will enable supervision in places where there aren’t accredited supervisors but there are competent people and GPs onsite. (Registrar)

It’s just been life-changing … before I came into this, medicine was such a hard slog and I had had a really difficult time like burning out through the hospital work and through that sort of on-call work. And there were times when I didn’t really even want to be a doctor, but having [supervisor A] and [supervisor B] has just changed all of that. It’s just been awesome. (Registrar)

Discussion

Remote supervision should ensure that it facilitates learning, ensures patient safety and assists the registrar in the development of professional identity.

Evaluation of the success of the model and remote practice guidelines involved working with the supervisors, the onsite supervision teams, the registrars and the practices, to assess whether the process of establishing the ‘right registrar, with the right supervision in the right location’¹⁰ was successful.

One of the key points identified is that flexibility is needed to contextualise the specific requirements of the practice and community with the skills of the registrar and supervisor. All sites and situations are different and need to be considered on a case-by-case basis. A process for tailoring and evaluating the needs and outcomes for the registrar, supervisor, practice and patients will ensure that the diversity of people and environments are well-matched, and that outcomes are safe.

The ability to contextualise the guidelines to the many different possible practice locations, registrar needs and supervisor requirements relies on their flexibility and the skillsets of the RACGP staff who are implementing them. Currently this is overseen by the original medical educators who are educating other staff to continue the process. This should also spread the burden of the time required to set up a remotely supervised placement.

The real success of remote supervision is seen when registrars stay in the community after completion of training or recommend their colleagues train with remote supervision.

Suggestions for improvement and identified challenges

During the final round of interviews, interviewees reflected on the overall placement and possible areas for improvement. They also reflected on challenges that arose due to environmental factors. These included:

• communication challenges between different organisations
• challenges associated with lack of certainty around registrar contract
• challenges associated with unexpected environmental issues such as floods
• high staff turnover at the practice creating a challenging training environment
• lack of cultural support.

Summary of recommendations:

• Commence the registrar interview process, and provide outcomes, as soon as possible to enable adequate planning for the placement and stability for the registrar.
• Provide early education and support for practices about the importance of timely confirmation of placement and signing of contract.
• Ensure the practice, the supervisor and the registrar each receives an orientation pack before the placement that contains essential materials.
• Support the training site and supervisor in contextualising the CRSPP scenarios and facilitate the interviews.
• Embed the cultural mentor into the onsite team and ensure their payment and support by the training site.
• Clarify and simplify the onsite team requirements and processes.
• Encourage the registrars and supervisors to be persistent in ensuring regular contact.

Changes to remote supervision guidelines after the evaluation:

• refined documents for training site and supervisor accreditation requirements and registrar requirements
• streamlined supervisor and registrar interview process, interview guide and outcomes document 
• streamlined risk management planning template and process 
• addition of an orientation checklist specific for remote supervision to assist the site and supervisor prepare and complete the orientation period
• addition of a remote supervision placement plan template to assist in planning and approving placements 
• more clearly articulated flexibility within the guidelines for application to different settings
• addition of funding principles to guidelines
• as the guidelines are to be applicable to all training programs, development of a separate internal-facing document outlining the process for staff to establish and manage an RACGP Australian General Practice Training remote supervision placement. 

Limitations

There is a concern that the medical educators who developed the guidelines also conducted the interviews. They discussed the process and results with the other authors throughout the research and regularly reflected on the possibility of any biases or conflicts. None of the authors had any jurisdiction over the registrars’ pathways or the day-to-day activity of the supervisors.

Conclusion

The evaluation of the pilot demonstrated that remote supervision placements utilising the guidelines were safe, supportive and provided a quality training experience. The evaluation was overwhelmingly positive, with only minor recommendations suggested. Both registrars are continuing in the locations in which they trained, and both reported that the remote supervision experience was ‘better than any face-to-face supervision’ they had experienced. The three supervisors were also keen to continue to remotely supervise registrars, and both training sites are likely to remotely supervise registrars again in the future.

The guidelines have been refined considering the pilot findings, and all processes and procedures will continue to evolve as the role of remote supervision in general practice training is further developed.

The guidelines have now been successfully implemented in many other training locations, all of which needed their own contextualised modifications to ensure success.

The final guidelines are now available at https://www.racgp.org.au/education/registrars/fellowship-pathways/policy-framework/handbooks-and-guides/remote-supervision/introduction.

Funding

No funding was received for this research.

Conflicts of interest

JB and TL wrote the guidelines that were then evaluated in this study, and all authors worked for the RACGP at the time of this research.