download PDF
Introduction
Health practitioners in many rural and remote communities experience inadequate timely professional support because of a shortage of on-site, experienced practitioners and geographic barriers1-3. Professional support refers to activities facilitating personal and professional growth, including professional supervision, mentoring, in-service training and practical assistance4,5. Research suggests that telehealth technologies can help less-experienced local staff, healthcare providers and patients in such areas access real-time, effective professional support and guidance from experts located elsewhere6-11.
The incorporation of augmented reality (AR) technology into telehealth systems has been well reported12. As an emerging human interface and telecommunication technology, AR offers users an immersive environment in which specific locations or activities in the real world are enhanced by computer-generated three-dimensional content13-15. AR-based telehealth systems have been shown to improve clinical outcomes, including reductions in medical errors, improvements in the quality of care and decision-making, and positive perceptions among relevant end users12,16,17.
In June 2020, given the complex issues resulting from the COVID-19 pandemic, the Vietnamese Government comprehensively reviewed its epidemic prevention plan. It approved a funding scheme for remote medical examination and treatment for 2020–202518. As a result, by September 2020, more than 1000 healthcare facilities were connected to nearly 30 central hospitals by teleconferencing systems19. Additionally, the applicability of advanced technologies in telemedicine systems has also been encouraged at all levels of the national healthcare system.
This study was funded by the Aus4Innovation program, managed by the Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia, an innovative partnership grant awarded to the University of Tasmania and a central tertiary hospital in Vietnam, to improve the usability of the situational telementorship between central medical experts (mentors) and rural junior practitioners (mentees) in managing challenging patient cases. For the first time, this initiative has transformed the way healthcare workers are assisted in rural and remote underserved areas in Vietnam. Using HoloLens v2 as the core AR device of the system, the mentor–mentee connection was established following the framework of situational telementorship20. With a heads-up display and hands-free video calling via the HoloLens headset, the remotely located mentee can connect with a centrally located mentor and share what they see through a video camera embedded in the HoloLens unit, helping them deal with clinical situations in real time and complete assessment tasks together. The mentor and mentee can also share reference materials, patient test results (eg X-ray or electrocardiograph), images or files in real time.
Study aim
The aim of the study was to evaluate the usability of AR technology for telementorship between central medical specialists and rural junior doctors in Vietnam, assess the perceived effectiveness of the mentoring relationship, and explore patient acceptability of AR technology during clinical consultations.
Methods
Location and setting
The AR remote assistance system included a mentor and eight mentee stations. The mentor station was placed at a large (more than 3500 beds) tertiary referral hospital (hereafter called the central hospital) in Hanoi, the capital city of Vietnam. Mentee stations were located in two provincial hospitals, four district health centres and two community health stations in Yen Bai Province (hereafter referred to as the health facilities) (Fig1). Yen Bai is a mountainous province in Northern Vietnam with a population of approximately 850,000, distributed across cities, towns and villages over nearly 7000 km2. The health facilities selected for the mentee stations were determined by the priorities of the national telehealth development scheme approved by Vietnam’s Ministry of Health in June 202018. The straight-line distance from the health facilities in Yen Bai to the central hospital ranged from 159 to 277 km. Difficult terrain and seasonal weather conditions in the province meant that road travel could take up to 8 h, making it difficult for patients to access specialist care.
At the central hospital, the mentor station is equipped with a laptop running Microsoft 365 Business Standard and includes the Microsoft Teams app. A mentee station set up at each mentee’s hospital includes a HoloLens v2 device with the Dynamics 365 Remote Assist app (called Remote Assist) licensed and activated. The laptops and HoloLens are connected to the local wi-fi network for internet access, with approximately 50 Mbps of bandwidth available at each station. The system's configuration has been published elsewhere21.
Figure 1: Locations of healthcare facilities in Yen Bai Province and their distance from Hanoi in North Vietnam.
User guides and teaching materials
A group of medical and AR experts from the University of Tasmania and the Vietnamese Central Hospital collaborated to produce user guides with step-by-step instructions in Vietnamese and English to facilitate the installation and operation of the AR system in Vietnam. As part of these materials, the scripts of six clinical emergency scenarios likely to be encountered in rural and remote populations in Yen Bai were developed, and a set of video clips was produced to demonstrate the use of the AR system in a clinical skills simulation laboratory (Fig2). The scenarios were acute pulmonary oedema, supraventricular tachycardia, acute hypertensive crisis, acute bronchial asthma, chronic obstructive pulmonary disease, and community-acquired pneumonia. Eight health professionals reviewed the scripts to ensure they complied with current treatment protocols in both countries and the framework of situational telementorship20. The recordings were then edited and dubbed in Vietnamese to produce video clips for the project's teaching materials.
Figure 2: Augmented reality remote assistance demonstration in clinical procedures in the simulation laboratory.
Participants
Medical specialists at the central hospital and medical practitioners at selected health facilities in Yen Bai Province were invited to serve as mentors and mentees, respectively.
Mentors were required to be experienced medical specialists, have been working at the relevant clinical units for more than five years and have experience working with rural doctors. The specialists were approached through a formal email invitation and, if interested, a follow-up phone call from the project coordinator to confirm availability.
The mentees were newly graduated or less experienced, interested in advanced technologies like AR, and willing to learn and use them. Information technology staff in the central and rural areas were also recruited to support the mentors and mentees technically during the training and rollout of the project.
Patients were recruited via poster advertisements at each of the eight participating health facilities in the province and were selected on the basis of their clinical condition and the likelihood that their doctor (the mentee) would benefit from a medical specialist’s (the mentor) advice in determining their care.
Emergency presentations were ineligible to participate because of patient safety concerns and the inability to delay urgent care, consistent with ethical guidelines that require new technologies to undergo rigorous testing under conditions that do not place patients at risk before they are applied in high-risk or life-threatening situations. Participation was voluntary, and informed consent was obtained from all participants.
Training sessions
Seven training sessions were delivered to provide mentors, mentees and technical staff with hands-on experience in using HoloLens and AR systems in clinical practice. The first five sessions were provided for technical staff, followed by one for the mentors at the central hospital and one for mentees in the province. The sessions were facilitated remotely by Australian educators, supported by trained Vietnamese technical staff. In each session, the Australian educators remotely presented the AR technology and guided participants in using the Remote Assist app on the HoloLens and the Teams app on the laptop. The Vietnamese technical staff were on site to support mentors and mentees during their practical training sessions, using Teams and Remote Assist to facilitate their connections. Following this, mentors used the device to connect with mentees remotely for additional training. Each session lasted from 1 to 3 h. While the mentors used Teams and the mentees used Remote Assist during subsequent patient consultations, all were trained in both tools to familiarise themselves with their use and potential for remote collaboration. No further training sessions were scheduled once all mentors and mentees confirmed their confidence in using the system and demonstrated the ability to operate the HoloLens and associated applications under supervision.
Intervention
Patient consultations were conducted in a suitably equipped private room at each facility. Each consultation was prescheduled for convenience and to accommodate patient and mentor availability. At each consultation, the mentee wore the HoloLens and activated the Remote Assist app to call the mentor using Teams (Fig3A). The mentor remotely assisted and guided the mentee in examining the patient and developing a treatment plan. The mentor could see and hear the patient, communicate with the mentee and share files or images via Remote Assist (Fig3B). The patient could not see the mentor but was aware of their presence during the consultation and could hear the mentee talk to their mentor.
Figure 3: A real-time remote consultation session. (A) The mentee communicates with the mentor using HoloLens 2. (B) The mentor instructs the mentee remotely.
Outcome measures
The study outcome measures were the mentors’, mentees’ and technical staff’s perceptions of the usability of the AR technology, mentorship effectiveness and patient technology acceptability.
The scale developed by Ingrassia et al22 and modified by the research team was used to measure the usability of the AR technology. Usability was interpreted as the system’s capacity to enable users to perform required tasks while supporting a positive and meaningful user experience23. The evaluation consisted of two sections. The first section incorporated the set-up usability scale developed by Brooke24, which aligned with the ISO 9241–400 guidelines regarding ergonomic factors and includes 17 items. According to Bangor et al25, this set-up usability scale was a highly reliable and flexible assessment tool for usability experts, with a Cronbach’s α of 0.911. The second section evaluated set-up input (15 items), set-up output (7 items), fidelity (8 items), immersion (4 items) and likeability (9 items). A Likert scale was used to rate each item from 1 (strongly disagree) to 5 (strongly agree). Mentors responded to 36 items while using a laptop rather than an AR device. In contrast, mentees using the AR device answered all 43 items.
With permission, the effectiveness of the mentor–mentee relationship was assessed by mentees using a 13-item scale developed by Berk et al26, with each item graded on a five-point Likert scale. The lead author of the scale granted the research team permission to use and modify the scale. Although publications and reports using this scale have not reported psychometric properties, it has been cited widely and used by medical and nursing teachers27. This may be due to its high face and content validity or the dearth of other psychometrically sound scales reported in medicine and the allied health literature28.
The mentor’s perception of the relationship was assessed using an instrument developed by Dimitriadis et al27. The modified instrument has five items, also graded on the five-point Likert scale. The research team added an item to obtain an overall perspective from mentors. At the end of the questionnaire, three questions allowed each mentor to provide personal feedback about the mentoring relationship.
The patient scale was designed to assess patient acceptability of doctors wearing headsets and using AR technology during consultations. The survey comprised 10 questions, selected on the basis of a review of studies on patient acceptability and wearable technologies29-31.
An expert panel reviewed all scales for face validity and to ensure that items were easy to understand, acknowledging that literacy levels would be low for some patients. Scales were then translated into Vietnamese and further reviewed by a group of Vietnamese medical and rural healthcare staff who were not participants in the study. This ensured semantic equivalence and cultural appropriateness while minimising the risk of meaning distortion. From the feedback provided, some minor modifications were made. These focused on contextual relevance (eg adapting terminology for telementorship scenarios). An open-ended question was added at the end of each scale to allow participants to provide a written response if they wished. The core constructs measured by the original scales were preserved. The final versions of all scales were back-translated to ensure the meaning of each item was consistent across languages.
The mentors, mentees and technical staff were requested to complete the AR usability and mentorship effectiveness scales at the conclusion of all consultations. The patients completed their scale at the end of their consultation.
Data analysis
Each dataset was numbered and manually reviewed for data completeness and accuracy, including checks for missing values, duplicate entries and out-of-range responses. Data analysis was conducted using SPSS v23.0 (IBM Corp.; https://www.ibm.com/products/spss-statistics). Categorical variable frequencies were depicted using charts, while continuous variables were assessed for mean and standard deviation. The analysis was descriptive, focusing on summarising usability scores, mentorship effectiveness ratings, and patient acceptability responses. Non-parametric tests were employed because of the lack of random sampling, the relatively small sample size, and the non-normally distributed results. In addition to quantitative analysis, open-ended comments were analysed using manifest content analysis, which summarises the explicit meaning of participants’ statements32,33. A total of 12 participants provided 12 free-text comments, comprising 8 mentees and 4 mentors (patients did not provide narrative comments). Illustrative quotations were selected based on representativeness, clarity and their ability to reflect typical views.
Ethics approval
This study received ethics approval from the Tasmanian Health Medical Human Research Ethics Committee (Project ID 23343) and the Bach Mai Hospital Medical Research Ethics Committee (Certificate No.759).
Results
Participants and consultations
Four mentors, 18 mentees, and four technical staff were involved in the study (Table 1). Two mentors were cardiologists and two were pulmonologists. The number of mentees was the highest at the provincial level (11), followed by the district (5) and community (2) levels. All mentees were medical doctors practising in the clinical wards. Three (75%) of mentors, 12 (67%) of mentees and 43 (46%) of patients were female. The four technical staff members were male.
Ninety-four inpatients, with an average age of 60.56 years (standard deviation 18.98), were consulted across 41 sessions over 4 weeks using the RA remote assistance system. Most sessions (83%) involved one mentee, while the rest (17%) involved two or three mentees consulting patients consecutively. An interruption occurred in nine consultations (22%) due to local severe weather and mentee shortages caused by COVID-19. All of these were rescheduled and completed successfully.
The district health centres accounted for nearly half of the patient consultations (45), more than provincial hospitals (26) and community health stations (23). The mentees working at community and district-level facilities saw, on average, more patients than those working at the larger provincial-level hospitals (11.5, 9 and 2.5 patients/mentee, respectively).
Patients had varying clinical conditions. Approximately 44% were complex cases with multiple conditions, such as diabetes combined with hypertension, heart failure and pneumonia, while the remaining patients had only a single condition. The most common recommendations from mentors for the patient were more individualised medication, clinical follow-up and referral for specialist care.
Table 1: Location and characteristics of participants
| Level of healthcare facility (n) |
Mentors n (%) |
Mentees n (%) |
Technicians n (%) |
Patients n (%) |
|---|---|---|---|---|
| Central – tertiary referral (1) | 4 (100) | 0 (0) | 3 (75) | 0 (0) |
| Provincial (2) | 0 (0) | 11 (61) | 1 (25) | 26 (28) |
| District (4) | 0 (0) | 5 (28) | 0 (0) | 45 (48) |
| Community (2) | 0 (0) | 2 (11) | 0 (0) | 23 (24) |
Augmented reality’s usability
The mentee response rate was 77.8%, while those of the mentors and technical staff were 100%. Table 2 presents the average scores of the AR’s usability by scale subtopics and participants. The statements relating to augmented reality usability for each subtopic, and mean scores, are shown in Appendix I. These scores were calculated after reverse-scoring negatively worded items. On a five-point Likert scale (1=strongly disagree, 5=strongly agree), scores above 3.5 indicate positive perceptions. All participants rated the usability of the HoloLens (for mentees and technical staff) and the system (for mentors) positively. Mentees reported favourable usability (mean=3.8±0.6), technical staff rated usability highly (mean=4.1±0.2), while mentors reported moderate usability (mean=3.1±0.2). Subscale analysis shows mentees rated situational awareness and effectiveness highest (means 4.0 and 3.9), suggesting confidence in real-time collaboration. Mentors’ lower scores reflect limitations in engagement and situational awareness when using laptops rather than headsets.
The mentors noted that they could observe live patients, engage in real-time conversations with mentees, and use the spatially organised medical records. One mentee remarked that the AR system helped all patients minimise their travel needs and associated costs. Instead of having to travel to a city hospital, the system enabled them to remain in their rural and remote communities while accessing specialist advice and support through their local healthcare providers.
It [the AR system] helps a lot because most of the patients at our health centre are either very young patients or older patients. For young patients, the financial situation is very difficult. For the elderly, their health conditions are not good. If people have to go from their place to the central hospital, it is nearly 200 km away. Given the patients’ poor health conditions, this is both difficult and expensive for them. (mentee)
Table 2: AR usability scores by subscale and participants
| Subtopic |
Mentees (n=14) |
Mentors (n=4) |
Technical staff (n=4) |
|||
|---|---|---|---|---|---|---|
| No. of items | Mean±SD | No. of items | Mean±SD | No. of items | Mean±SD | |
| General use of HoloLens/the system | 14 | 3.6±0.4 | 8 | 3.4±0.4† | 14 | 3.6±0.7 |
| Interaction with HoloLens/the system | 9 | 3.7±0.6 | 9 | 3.3±0.4 | 9 | 4.2±0.7 |
| Display of HoloLens | 9 | 3.9±0.6 | NA | NA | 9 | 4.1±0.3 |
| Situational engagement | NA | NA | 4 | 2.9±0.1 | NA | NA |
| Situational awareness | 3 | 4.0±1.0† | 5 | 2.7±0.1 | 3 | 4.7±0.4† |
| Effectiveness | 5 | 3.9±1.0 | 5 | 2.9±0.3 | 5 | 4.7±0.2† |
| Likeability | 3 | 3.6±0.7 | 3 | 3.1±0.3 | 3 | 4.0±0.3 |
| Total | 43 | 3.8±0.6 | 34 | 3.1±0.2 | 43 | 4.1±0.2 |
† Highest/equal highest mean score within participant group, highlighting comparatively stronger subscale performance.
NA, not applicable. SD, standard deviation.
Mentorship effectiveness
Table 3 and Table 4 present the mentorship effectiveness scores for four mentors and 14 mentees. The statements relating to mentorship effectiveness for each subtopic, and mean scores, are shown in Appendix II. The overall means were calculated after reverse-scoring negatively worded items. Overall, mentorship effectiveness scores were positive for both groups. Mentees rated their mentors highly (mean=4.2±1.1), particularly for communication and support (items 1 and 5). Mentors also reported satisfaction (mean=3.8±0.3), indicating that the telementorship model was effective despite technical and logistical challenges.
The mentees rated the highest average score for satisfactory communication with their mentors (Table 4, item 1) and the support and encouragement of the mentors (Table 4, item 5). Most mentees were satisfied with their mentors. Some mentee feedback was that ‘the project means a lot to them’ and ‘they expected that the project could last longer to have more consultations’.
Participants’ comments reflected positive experiences of support, increased confidence, and practical challenges when using the AR system. For example, mentees noted feeling ‘more confident making decisions’, while mentors highlighted that the system allowed them to ‘guide examinations more effectively’. Some also described challenges such as limited availability of imaging, noting that ‘not all features could be used when X-rays were unavailable’.
Yesterday, a patient visited us. The patient had many health issues that were not easy to diagnose and then manage. After connecting with the doctor at the Hospital, we had a clear direction for diagnosis and treatment. (mentee)
The stress of managing the patients in rural areas was also recognised by a mentor at the central hospital. They mentioned that their colleagues in rural hospitals had to treat severe cases and were under high pressure because of fewer experienced colleagues on site to guide them. According to the mentor, the telementorship improved the situation for their colleagues in rural areas.
When there is a group working together, it will have a better direction for patient management. Mentally, I think they [rural practitioners] will be more stable and more confident, and in the future, when they encounter similar situations, they can handle them better. (mentor)
The mentee supports the effectiveness of the telementorship.
After being discussed and supported by experts of the central hospital, the health situation of most of the patients we treat has improved. (mentee)
Mentors highlighted positive aspects of the AR system, including providing specific consultations for difficult or complex cases, giving professional advice on clinical examination and analysis of test results, and strengthening connections between central and rural medical staff. Negative aspects were also mentioned, including a limited number of consultants available and their availability due to busy schedules, and the limited effectiveness of using all the features of the HoloLens at the communal health centres because of the limited availability of diagnostic results. Mentors also mentioned the learning aspects of mentoring relationships, such as feeling as if they were at the scene with the mentee and an improved understanding of the actual situation of healthcare service delivery at the district and communal health centres.
Table 3: Mentor assessment of mentorship effectiveness (n=4)
| Item | Mean±SD |
|---|---|
| 1. My mentees were well-matched to my skills and experience | 4.5±0.6 |
| 2. My mentees were difficult to communicate with | 3.0±1.4 |
| 3. I was able to answer my mentees’ questions satisfactorily | 4.0±0.0 |
| 4. I helped provide direction and guidance to my mentees | 3.8±0.5 |
| 5. I have had a positive impact on my mentees’ performance | 3.8±0.5 |
| 6. Overall, I was very satisfied with the mentoring relationship | 3.8±0.5 |
| Overall mean score | 3.8±0.3 |
SD, standard deviation.
Table 4: Mentee assessment of mentorship effectiveness (n=14)
| Item | Mean±SD |
|---|---|
| 1. My mentor was not difficult to communicate with | 4.5±0.9 |
| 2. My mentor demonstrated professional integrity | 4.2±1.4 |
| 3. My mentor demonstrated content expertise in my area of need | 4.2±1.4 |
| 4. My mentor was responsive to my needs | 4.2±1.1 |
| 5. My mentor was supportive and encouraging | 4.5±1.1 |
| 6. My mentor provided constructive and useful critiques of my work | 4.4±1.1 |
| 7. My mentor motivated me to improve my work | 4.2±1.1 |
| 8. My mentor helped provide direction and guidance | 4.2±1.4 |
| 9. My mentor answered my questions satisfactorily | 4.1±1.4 |
| 10. My mentor acknowledged my contributions appropriately | 4.1±1.4 |
| 11. My mentor suggested appropriate resources | 3.9±1.3 |
| 12. My mentor challenged me to extend my abilities | 4.4±1.1 |
| 13. Overall, I was satisfied with my mentor | 4.1±1.4 |
| Overall mean score | 4.2±1.1 |
SD, standard deviation.
Patient perceptions
Patient responses show strong acceptability of the AR technology. Ninety-three out of 94 patients (98.9%) completed the survey (Table 5). The survey consisted of 10 items and demonstrated a moderate level of internal consistency (Cronbach’s α=0.6). For most patients (about 71%), this was the first time they had a consultation with a doctor wearing a headwear device. They reported that the doctors clearly explained the device’s use in more than 80% of cases. The percentage of patients (70.3%) not distracted by the headwear device was slightly more than that of the doctors (61.5%). Most patients (nearly 85%) found it easy to communicate with their doctor while wearing the device. Notably, more than half (51.6%) were happy with their doctor wearing the device. No patient reported sharing private information via the system as a concern. Most patients (66.3%) enjoyed the experience, and 75% would be happy for their doctor to use the headset again. These findings suggest that AR consultations were well received and did not negatively impact communication or trust.
Notably, the mentee observed that the AR system made patients feel more confident and more trusting of local practitioners and health facilities.
The fact that when I explained about the use of technology to the patients, they were very excited, [because] of being connected to the doctors at the central hospital. Patients were also very confident and believed in the doctor at their local health centre. (mentee)
Table 5: Patient device acceptability responses
| Item | Yes | No | Partly Yes or No, or no feeling either way | Total |
|---|---|---|---|---|
|
n (%) |
n (%) | n (%) | n (%) | |
| 1. Has a doctor examined you wearing a device before? | 22 (23.7) | 66 (71) | 5 (5.3) | 93 (100) |
| 2. Did the doctor clearly explain the use of the headwear device to you? | 77 (82.8) | 0 (0) | 16 (17.2) | 93 (100) |
| 3. Were you distracted by the doctor wearing the headwear device? | 6 (6.6) | 64 (70.3) | 21 (23.1) | 91 (100) |
| 4. Was the doctor distracted by the headwear device? | 2 (2.2) | 56 (61.5) | 33 (36.3) | 91 (100) |
| 5. Was it difficult to communicate with the doctor when they were wearing the headwear device? | 5 (5.4) | 87 (94.6) | 0 (0) | 92 (100) |
| 6. Are you happy with the doctor wearing the headwear? | 48 (51.6) | 1 (1.1) | 44 (47.3) | 93 (100) |
| 7. Were you concerned about sharing your private information when the doctor was wearing the headwear device? | 0 (0) | 43 (47.3) | 48 (52.7) | 91 (100) |
| 8. Did you trust the doctor because of wearing the headwear device? | 43 (46.7) | 2 (2.2) | 47 (51.1) | 92 (100) |
| 9. Would you be happy for your doctor to wear the headwear device again? | 69 (75.0) | 2 (2.2) | 21 (22.8) | 92 (100) |
| 10. Did you enjoy the overall experience of being examined by your doctor wearing a headwear device? | 61 (66.3) | 2 (2.2) | 29 (31.5) | 92 (100) |
Discussion
This study evaluated the usability of AR technology for telementorship between central medical specialists and junior doctors in rural and regional areas of Vietnam, assessed the effectiveness of mentorship, and explored patient acceptability during doctor–patient consultations. Usability was assessed across multiple dimensions, including interaction, effectiveness and situational awareness. The results indicate that usability was rated positively by mentees (mean=3.8) and technical staff (mean=4.1), whereas mentors rated it as moderate (mean=3.1), reflecting differences in device interaction. Mentorship effectiveness was rated highly by both mentors (mean=3.8) and mentees (mean=4.2), suggesting that AR-supported telementorship facilitated effective communication and guidance. Patient acceptability was strong, with 66.3% reporting satisfaction and 75% willing to engage in future consultations using AR technology. These results demonstrate that AR telementorship is feasible and well received in rural clinical settings, aligning with the study’s aim to assess usability and adoption factors in real-world practice.
The positive usability ratings and strong patient acceptability observed in this study complement other research demonstrating the feasibility of AR telementorship for enhancing remote clinical support. Systematic reviews and experimental studies consistently report that AR-based telementoring improves task performance, reduces errors and increases practitioner confidence compared to traditional remote guidance methods12,34. These benefits are particularly relevant in rural contexts, where geographic isolation and workforce shortages limit access to specialist expertise. Our findings corroborate earlier work showing that AR systems enable real-time visual and verbal interaction, fostering situational awareness and effective collaboration between mentors and mentees35,36. Furthermore, the high levels of patient acceptability observed in this study echo emerging evidence that immersive technologies can enhance trust and satisfaction in telehealth encounters, provided that communication remains clear and culturally appropriate37. Collectively, these results reinforce the growing consensus that AR-supported telementorship is a viable strategy for bridging professional support gaps in rural health care and complements broader telehealth initiatives aimed at improving equity and quality of care38.
In many rural and remote areas, healthcare staff have difficulty accessing real-time, effective assistance and realistic training from experts located centrally. In addition, the COVID-19 pandemic has affected all sectors of global society, imposing severe restrictions on individuals’ mobility and access to education, services and health care. This scenario presented a unique opportunity to escalate investment in and adoption of AR head-mounted display technology39,40. Contributing to the literature in this research field, this study presented findings on the usability of HoloLens 2 for remote assistance to medical practitioners in rural clinical environments in Vietnam for the first time, with a significantly larger number of patients than in other recently reported studies41-45. Furthermore, patient consultations were conducted in eight rural healthcare facilities in a mountainous province, including very remote communities. The implementation of the AR system involved 18 practitioners in the province and four specialists at a central hospital in Hanoi. It was the first time these medical staff used such technology in real-time patient consultations.
In Vietnam, the AR remote assistance systems can enhance teleconferencing systems established across health facilities nationwide. In these systems, a teleclinic office is equipped in each health facility with large screens, professional webcams, computers with a dedicated high-speed internet connection, and videoconferencing software. Rural practitioners prepare presentations on patients in advance to present to and consult with specialists in Hanoi. However, the established system’s lack of mobility is apparent, making real-time consultation services in the patient room almost impossible with such teleconferencing systems. The findings of this study showed that the AR system deployed can provide greater mobility for communication and an enhanced alternative to the current setup. Using a HoloLens headset, rural practitioners can visit patients in the clinic or their rooms and communicate with a central specialist from that location.
The AR technology provides technical advantages over other visual–auditory telecommunication technologies, including teleconferencing systems12. Wearing HoloLens headsets, practitioners in Yen Bai could visit ward rooms to check on patients in need in real time. Through the AR system, these rural practitioners shared with the central specialists what they saw while keeping both hands free to perform patient assessments or clinical treatment procedures. They also shared the patient’s X-ray or CT scan results, which the specialists can annotate for notifications or instructions. The specialists shared treatment protocols as PDF files, displayed spatially in the practitioner’s space, enabling interaction. Although this study did not count the number of such interactions via the AR system, the findings showed evidence of using the AR features in assisting and instructing junior practitioners remotely in a clinical environment in Vietnam.
Unlike studies conducted in more controlled environments, such as operating theatres and simulation laboratories, implementing AR technology in healthcare facilities revealed a few limitations. For safety reasons, the rollout into clinical areas was conducted with scheduled doctor–patient consultations rather than in response to at-call emergencies, and no clinical procedures or surgeries were performed during the consultations. The application of the solution across all scenarios will require further training of additional personnel and the organisation of resources, such as the 24/7 availability of relevant consultants and information technology support, which is a potential topic for further study. It is worth noting that some remote consultations did not take place as planned due to power outages caused by severe weather and a shortage of health workers in remote areas. Additionally, patient imaging examination results (eg X-rays and CT scans) were also not always available or were not in a clear, readable form in some healthcare facilities in the province, especially at district and communal levels, which limited the full utilisation of AR features on the system. Additionally, the logistics of the communication via the AR system made it more complicated to connect multiple local practitioners in the discussion.
Strengths and limitations of this study
For the first time, an AR telementoring system has been developed at an Australian university and used in a series of central, rural and remote healthcare facilities across Vietnam. The study provided evidence of the successful implementation of AR technology for situational telementorship, enabling central specialists to offer real-time consultations to junior rural practitioners caring for patients. The intervention involved a substantial number of patients (94) at healthcare facilities within various levels of the national healthcare system.
The study measured the overall perspectives of all participant groups regarding technology and telementorship, achieving a significant response rate. The findings demonstrate the usability of AR technology and enhance the clinical support available to doctors in remote locations, effectively bridging the gap in practical skills and knowledge between central and rural healthcare facilities.
Alongside the study’s strengths, some limitations were noted. Although the technical training provided was limited due to COVID-19 restrictions and resource constraints, all mentors and mentees confirmed their confidence in using the AR system before participating in consultations. Informal checks were conducted during training sessions, where participants demonstrated the ability to operate the HoloLens and associated applications under supervision. Although no formal competency assessment was conducted, the successful completion of 41 consultations without major technical errors suggests that participants achieved sufficient proficiency to meet the study objectives. Future research should incorporate structured evaluations of user competence to strengthen validity and ensure optimal utilisation of AR features in clinical settings. Although a comprehensive training package was developed, additional sessions were necessary for mentors and mentees to become fully comfortable with the technology’s features. In future studies, additional training could further boost user confidence and ability to apply the AR features during consultations.
Moreover, while a significant number of patients were involved, the small number of specialists (4) and practitioners (14) in internal medicine makes it challenging to generalise the findings about the usability of AR technology to other settings. Another limitation was the lower response rate among mentees (77.8%) than among mentors and technical staff (100%). This discrepancy was largely due to contextual challenges in rural settings, including competing clinical duties, staff shortages and disruptions caused by severe weather and COVID-19 restrictions. While the achieved rate was sufficient for the study objectives, it may have introduced some bias and limited the completeness of mentee perspectives.
In addition, the study did not report evidence on consultation outcomes due to concerns about patient confidentiality and difficulties accessing patient records during the COVID-19 pandemic. Many health facilities implemented strict infection-control measures and restricted movement within hospitals, thereby limiting researchers’ ability to collect detailed clinical data. Additionally, the primary aim of this study was to evaluate the usability of AR technology and explore patient acceptability, rather than measure clinical effectiveness or error rates. Future research should incorporate outcome-based measures to assess whether AR-supported telementorship improves diagnostic accuracy and treatment and reduces clinical errors.
Conclusion
By evaluating the usability of AR technology for telementorship between central medical specialists and rural junior doctors in Vietnam, this study addresses the gap in existing professional support between central and rural health facilities in Vietnam. It provided evidence of patient acceptance and usability of the technology by mentors and mentees for situational telementorship. The findings also highlighted the experience and usability of implementing an AR system in actual clinical environments.
Acknowledgements
The authors would like to thank Dr Lan Anh Hoang and Dr Thi Hong Van Le in Yen Bai Province, and Hong Phong Le, Ngoc Chien Hoang, and Tien Thanh Nguyen at the Training Centre, BMH, for their support and participation in the data collection for this study. The authors also thank Darren Grattidge, Nga Tran and Christine Low at the Centre for Rural Health, University of Tasmania; Kevin Wilmore, Margaretha Yam and nursing lecturers at the Simulation and Clinical Education Centre, University of Tasmania; Van Thuong Bui and Anh Duc Hoang at the BMH; and Amanda Carnicelli and Kahlia Smith for their support in clinical scenario development and system testing. The authors acknowledge support from the University of Tasmania, Bach Mai Hospital, the Yen Bai Provincial Health Department, and the province’s health facilities.
Funding
This work was supported by the Aus4Innovation program funded by the Commonwealth Scientific and Industrial Research Organisation, Australia (C031915). The funders had no role in the conduct of the research or writing of the article.
Conflicts of interest
The authors declare no conflicts of interest.
AI disclosure statement
The authors used Grammarly for language refinement; all content was verified by and is the responsibility of the authors.