Project Report

The Cumbria Rural Health Forum: initiating change and moving forward with technology

AUTHORS

Jae-Llane Ditchburn1 PhD, Research fellow *

Alison Marshall2 PhD, Professor

CORRESPONDENCE

* Jae-Llane Ditchburn

AFFILIATIONS

1 University of Cumbria at Energus, Lilyhall, Workington, Cumbria, UK

2 Cumbrian Centre for Health Technologies (CaCHeT) Faculty of Health & Science, University of Cumbria, Cumbria, UK

PUBLISHED

7 June 2016 Volume 16 Issue 2

HISTORY

RECEIVED: 1 October 2015

REVISED: 11 May 2016

ACCEPTED: 24 May 2016

CITATION

Ditchburn J, Marshall A.  The Cumbria Rural Health Forum: initiating change and moving forward with technology. Rural and Remote Health 2016; 16: 3738. Available: www.rrh.org.au/journal/article/3738

AUTHOR CONTRIBUTIONS

© James Cook University 2016


abstract:

Introduction: The Cumbria Rural Health Forum was formed by a number of public, private and voluntary sector organisations to collaboratively work on rural health and social care in the county of Cumbria, England. The aim of the forum is to improve health and social care delivery for rural communities, and share practical ideas and evidence-based best practice that can be implemented in Cumbria. The forum currently consists of approximately 50 organisations interested in and responsible for delivery of health and social care in Cumbria. An exploration of digital technologies for health and care was recognised as an initial priority. This article describes a hands-on approach undertaken within the forum, including its current progress and development.
Methods: The forum used a modified Delphi technique to facilitate its work on discussing ideas and reaching consensus to formulate the Cumbria Strategy for Digital Technologies in Health and Social Care. The group communication process took place over meetings and workshops held at various locations in the county.
Results: A roadmap for the implementation of digital technologies into health and social care was developed. The roadmap recommends the following: (i) to improve the health outcomes for targeted groups, within a unit, department or care pathway; (ii) to explain, clarify, share good (and bad) practice, assess impact and value through information sharing through conferences and events, influencing and advocacy for Cumbria; and (iii) to develop a digital-health-ready workforce where health and social care professionals can be supported to use digital technologies, and enhance recruitment and retention of staff.
Conclusions: The forum experienced issues consistent with those in other Delphi studies, such as the repetition of ideas. Attendance was variable due to the unavailability of key people at times. Although the forum facilitated collective effort to address rural health issues, its power is limited to influencing and supporting implementation of change. Within the implementation phase, the forum has engaged in advising and facilitating policy change at all levels. Thus, the forum has become a voice to influence change towards the advancement of health and social care through digital technologies. The forum continues to serve as a think tank and influencer for change in rural health and social care issues in Cumbria. The forum has increased awareness of digital health and social care solutions, mapped best practice and developed a digital strategy for health and social care in Cumbria.

Key words: digital technology, e-health, England, health services, needs and demand, social care, strategy, telecare, telehealth, telemedicine.

full article:

Introduction

Background

Cumbria is the second largest county in England by area. Its population, at 495 000, makes it one of the most sparsely populated. Half of Cumbria's total population live in areas that would formally be defined as rural1, and the urban centres are relatively small and geographically dispersed2. This rurality is not unique to Cumbria and is known to cause specific problems in the delivery of health and social care3-5.

Accessibility to key local services in Cumbria (including GP appointments and hospitals) currently exceeds national average times in the UK2. Delays in hospital admissions due to administrative procedures have previously been documented3. The high rate of suicide amongst farmers in remote areas in Cumbria initiated efforts to encourage people to seek help for their mental and physical health and minimise the relative inaccessibility of primary care, particularly in remote areas4. For example, an outreach model was designed to deliver general health care to farmers by using nurse practitioners5. Another problem has been the risk of hospital closures due to a recruitment crisis faced by local authorities in Cumbria6,7.

The Cumbria Rural Health Forum was formed in September 2013 to address the following questions related to the delivery of health and social care in Cumbria:

  1. What does good rural health and social care look like?
  2. How can digital technology address some of the issues?

Initial forum meetings built a consensus for a program of work to develop a strategy for future implementations of digital health and social care technologies based on issues relevant to Cumbria. In April 2014, the Cumbria Rural Health Forum received funding from the Academic Health Science Network for the North East and North Cumbria to enable a research team from the University of Cumbria to be engaged. This article aims to provide a concise description of the Cumbria Rural Health Forum, detailing its work, current progress and future intentions.

Aims and priorities

The forum aims to improve health and social care delivery for rural communities, and share practical ideas and evidence-based best practice that can be implemented in Cumbria.

The priorities of the forum during 2014-2016 are:

  • to explore the value of digital technologies in health and social care, including telehealth, telemedicine, telecare and assistive technologies, and e-health
  • to influence and provide evidence to implement changes in health and social care
  • advocacy and representation.

Membership

The forum comprises a varied membership involving health and social care providers from the public, private and third sectors, digital technology companies, health and care social commissioners and policy makers in Cumbria. Membership started at 30 in 2013 and has increased to 158 members from 56 organisations from private, public and third sector agencies. Membership is open to any organisation or representative body that is interested in contributing to the forum's mission - either based in Cumbria, providing health and/or social care services to Cumbria, or planning to do so.

Methods

The Delphi method

The forum adapted the Delphi method8,9 for conducting its work through meetings and workshops between stakeholders to reach agreement toward agreed actions, tasks and issues in discussion (Fig1). The Delphi method was originally developed as an interactive consensus-orientated technique relying on a group of experts answering collected data from respondents in rounds10,11. It has been used in various fields of study such as project management12, geriatrics13, operational research14, social policy15,16 and information systems17,18.

While the Delphi method is normally survey-based, the method was adapted in the present study to be used in group settings11, combined with the open innovation approach19,20. Open innovation is 'the use of purposive inflows and outflows of knowledge to accelerate internal innovation, and to expand the markets for external use of innovation'21. It encourages the use of internal and external ideas, reflected in the sharing of best practice relevant to Cumbria, and making connections with both internal and external resources22. The following modifications were made to the Delphi technique used by the forum:

  • group discussions through forum meetings, including feedback on email and discussions on social media
  • discussion and generation of structured statements at roadmapping workshops
  • incorporating elements of open innovation where the sharing of best practice with forum members was held to set the context for them to fully understand the purpose and aims of the forum. Forum members were also invited to answer questions and to work together on specific exercises relevant to the issues in discussion.

In the whole-group modified Delphi method used by the forum, members worked in groups, either answering questions or carrying out implementation exercises, and reported back to the full meeting, using flip charts and verbal feedback. A brief session of questions then followed for clarification. In addition to the group work, there was sharing of best practice through presentations on digital health and social care issues, services or projects, either locally or outside Cumbria. Information about what went on during forum meetings and workshops was emailed to all members. This updated people who were unable to attend meetings and enabled them to send any comments, thoughts or feedback by email. The forum used Twitter to conduct four chats on digital health technologies. All information was collected and sorted into lists to be fed back to the Delphi rounds.



Figure 1: Adapted Delphi technique used by the Cumbria Rural Health Forum.

Issues relevant to Cumbria

The first stage of the process was to confirm a common understanding of the rurality issues that were considered to affect delivery of health and social care services to the Cumbrian population:

  • dispersed communities, meaning that people have limited access to services and have to travel further to access basic health care
  • smaller GP practices and other health centres, meaning that staff may feel professionally isolated and removed from opportunities for professional development
  • a greater reliance on volunteer services
  • a population that comprises relatively more older people than in urban centres
  • poor quality broadband and mobile infrastructure in Cumbria.

The first forum workshop also confirmed that the use of digital technologies in improving health and social care in Cumbria was a key priority for the forum (see Appendix I for the outline of forum events).

It was agreed at the outset that the strategy work should not be limited to any particular technology. Digital health terminologies have been used differently in the literature by different authors23-25. To clarify the subtle differences in these terminologies and avoid confusion, the forum adopted agreed definitions of terms (Fig2). It was agreed to exclude technologies in development (eg new types of wearable sensors) and the use of digital technology for purely administrative purposes (such as online appointment booking).

The program of work was planned around the following objectives:

  1. Map existing digital health and social care practice within Cumbria and transferable best practice from elsewhere.
  2. Gain understanding of issues specific to rural health and social care.
  3. Identify needs and opportunities for use of digital technologies (eg for remote consultations).
  4. Develop a roadmap for implementation within Cumbria, including roles of key organisations, implementation plans.



Figure 2: Agreed scope and terminology used by Cumbria Rural Health Forum.

Ethics approval

Part of the original work described in this article were two qualitative original research studies, which were granted ethics approval by the University of Cumbria Research Ethics Committee (ref. 14/08, involving general practitioners and practice managers; ref. 14/10, involving people with long-term health conditions). These studies were carried out to investigate the role of digital technologies from the perspectives of healthcare professionals (GPs) and patients (appendixes II and III).

Results

The findings are available on the forum website26.

Working on the Cumbria Strategy for Digital Technologies in Health and Social Care

The program was undertaken over a 15-month period, shown on the timeline in Figure 3. The forum's work is presented in Appendix I. An investigative program underpinned the digital roadmapping workshops to gather necessary data to inform the forum and provide a basis for discussion27,28.

  • To map existing digital health practice in Cumbria, forum members shared information on projects or services currently available in the county, including those that had expired. A total of 27 digital health and social care activities were reported. A full report is available online29. Informal interviews with members of staff working on the digital health projects or services allowed the research team to understand work conditions, staff experience and how the services were provided.
  • A literature review on digital health and its application in international rural areas was carried out. The literature review found similarities to Cumbria in problems accessing health care experienced by people living in remote areas 30,31, workforce issues32-34 and use of digital health technologies35-38, which are either being planned or have recently been implemented39-41. Other issues were socioeconomic status of rural communities42-44, mental health problems45-49 and the implications of digital technologies for managing long term conditions40,50-52. A report on the literature review is available online53.
  • Two qualitative studies involving practice managers (n=5), general practitioners (n=15) and people with long term health conditions (n=25) were examined. Further information on the studies is available online26. The studies indicated that:
    • telephone triage was increasingly being adopted by general practices as a means for patients to access healthcare. When asked about using digital technologies for consultations, practice managers recognised the potential benefits of remote consultations
    • general practitioners believed that digital technologies could help reduce demand for their involvement in minor illnesses through improving transactional efficiency in the provision of services. Digital technologies in social media could be used to educate and empower patients
    • people with long term health conditions expressed willingness to use remote consultations if it was made available to them, and welcomed the prospect of email or secure messaging with their care providers. Researchers found a lack of knowledge of possibilities, such as access to electronic health records.

All findings were synthesised and fed back into the forum through the roadmapping workshops for members' views. Consensus was reached by the forum on the following:

  • The international literature review reflected similarity of experience relevant to Cumbria.
  • The mapping work found that efforts to adopt digital technologies have already begun in Cumbria, but there were highlighted areas for improvement.
  • Both professionals and patients recognised the potential benefits of integrating digital technology into health and social care in Cumbria.
  • It was recognised that staff need training and skills development specific to digital care services and technology for successful implementation.
  • All digital technologies reviewed were thought to offer some benefit to rural communities, particularly where they can be used to avoid travel by either a patient or care professional.



Figure 3: Cumbria Rural Health Forum timeline.

Finalising the strategy

Three major themes were finalised for the roadmap on implementing the Cumbria strategy for digital technologies in health and social care (Fig4):

Improve health outcomes through use of digital technology: Successful planning needs to focus around particular problems or care pathways. The implementation plan will proceed through thematic workshops linking small groups of professionals working on a single pathway, patient group or service.

Explain, clarify, share good (and bad) practice, assess impact and value: The evidence base now assembled supports a growing role in influencing and advocacy. The dissemination, communication, information sharing and collation, networking and discussion activities will continue.

Develop a digital-health-ready workforce: The forum will work on defining a set of minimal required skills for digital health and social care, to enable design and delivery of services in clinical settings54. The use of e-learning and video-conferencing for training and development will be considered55,56 .



Figure 4: Roadmap for Cumbria Strategy for Digital Technologies in Health and Social Care.

Lessons learned and implications

Findings from the forum have significant implications for practice, where multi-organisational consensus building is planned. The forum experienced issues consistent with those of other Delphi studies57. The number and representativeness of people at Delphi rounds affect the potential for ideas and how evaluation and consensus is reached58. There were some irregularities in meeting or workshop attendance. Despite holding a membership of more than 100 individuals, attendance at meetings was in the range of 20-30 people. The exact representation of stakeholder organisations also varied. To ensure full participation and involvement of all stakeholders, open-ended feedback by email was accepted from stakeholders who could not attend forum events. Stakeholders were encouraged to provide additional comments or express disagreement.

The next issue was not being able to engage key people during discussions - particularly GPs, who were frequently unable to attend due to clinical commitments. To ensure that their views could be received, Twitter was used to hold live discussions on digital health topics. To date, four Twitter chats have been held (available at https://storify.com/CumbriaRHF/crhfchat). The forum's Twitter chats generated keen interest in future collaborative efforts, particularly at primary care level (Appendix IV).

The forum experienced issues common to group problem solving59, such as 'group rut' where a group seems to have the same discussion repeatedly, regardless of the current agenda. To remedy this group rut, forum participants were reminded of the goals of the group work or exercise in question. Structured exercises were used during workshops60. Poor follow-through61 was also experienced amongst a few stakeholders, where they had not fully understood the purpose of a particular meeting or workshop.

To gain understanding and foster familiarity in working together as a forum, participants were reminded of previous discussions and updated on investigative work. This helped to encourage interest, ownership and active participation amongst stakeholders62.

The forum facilitated collective effort to address rural health issues, but it does not have any mandate to deliver change directly. It is emerging as an important think tank and influencing body.

Forum events provided opportunities for the sharing of best practice, discussion of health and social care issues and networking (Fig5). More significantly, the forum has influenced emerging policy within the Cumbria health economy and discussions are in progress to formulate an ongoing advisory role to senior policy and decision makers.



Figure 5: Overview of Cumbria Rural Health Forum roles, activities and interactions.

Conclusions

The forum has been successful in galvanising the interest and support of more than 50 organisations and more than 150 professional individuals in Cumbria, concerned with exploring important questions of mutual interest. The involvement of the voluntary or third sector and the private sector and dialogue with public sector bodies has been a particular achievement. The modified Delphi process adopted has presented some difficulties and limitations, but has ultimately led to a shared implementation plan, which has been successful in securing additional funding support. Awareness has been increased of digital health and social care solutions and a body of evidence has been compiled. The forum is recognised as an important think tank and influencing body, with a wider dissemination and advocacy role.

Acknowledgements

The Cumbria Rural Health Forum was funded in this phase of its work by the Academic Health Science Network for North East and North Cumbria. Further funding for the implementation phase was secured from that network and from the North West Coast Academic Health Science Network. In-kind support has been provided by members, in particular by Cumbria Partnership Foundation National Health Service Trust, Cumbria Clinical Commissioning Group, Cumbria County Council, Action for Communities in Cumbria and University of Cumbria. The authors would like to thank Tom Bell for his involvement in the mapping work, engagement of health professionals and comments on the manuscript; Keith Jackson for his input in the literature search; and Peter Knock for his feedback on the manuscript. The authors are grateful for the support received by everyone who has participated in the forum events or contributed in one way or another.

References

1. Department for Environment, Food and Rural Affairs. Rural urban definition. (Internet) 2014. Available: https://www.gov.uk/government/collections/rural-urban-definition (Accessed 14 July 2014).

2. Cumbria Intelligence Observatory. Transport, connectivity and accessibility. (Internet) 2015. Available: http://www.cumbriaobservatory.org.uk/economy/Transport.asp (Accessed 9 September 2015).

3. Herd EB. Terminal care in a semi-rural area. British Journal of General Practice 1990; 40(335): 248-251.

4. Gregoire A. The mental health of farmers. Occupational Medicine 2002; 52(8): 471-476.

5. Walsh M. A nurse practitioner-led farmer's health service: setting up and evaluating a UK project. Australian Journal of Rural Health 2000; 8(4): 214-217.

6. The Bay and Aiir. Millom Hospital will not close, says NHS Trust. (Internet) 2014. Available: http://www.thebay.co.uk/news/local-news/millom-hospital-will-not-close-says-nhs-trust (Accessed 13 September 2015).

7. The Whitehaven News. West Cumberland Hospital's A&E could close next year. (Internet) 2015. Available: http://www.whitehavennews.co.uk/news/West-Cumberland-Hospitals-AundE-could-close-next-year-2b0d9d5b-60f6-4373-8ab7-dd619e27bd09-ds (Accessed 5 December 2015).

8. Dalkey NC, Brown BB, Cochran S. The Delphi method: an experimental study of group opinion. Santa Monica: Rand Corporation, 1969.

9. Linstone HA, Turoff M. Delphi method: techniques and applications. Boston: Addison-Wesley, 1975.

10. Hsu C, Sandford BA. The Delphi technique: making sense of consensus. Practical Assessment, Research & Evaluation 2007; 12(10): 1-8.

11. Dick B. Delphi face to face. Resource papers in action research. (Internet) 2000. Available: http://www.aral.com.au/resources/delphi.html (Accessed 14 August 2015).

12. Brill JM, Bishop M, Walker AE. The competencies and characteristics required of an effective project manager: a web-based Delphi study. Educational Technology Research and Development 2006; 54(2): 115-140.

13. Robinson KR, Leighton P, Logan P, Gordon AL, Anthony K, Harwood RH, et al. Developing the principles of chair based exercise for older people: a modified Delphi study. BMC Geriatrics 2014; 14(1): 65.

14. Akkermans HA, Bogerd P, YĆ¼cesan E, van Wassenhove LN. The impact of ERP on supply chain management: exploratory findings from a European Delphi study. European Journal of Operational Research 2003; 146(2): 284-301.

15. Critcher C, Gladstone B. Utilizing the Delphi technique in policy discussion: a case study of a privatized utility in Britain. Public Administration 1998; 76(3): 431-449.

16. Buck AJ, Gross M, Hakim S, Weinblatt J. Using the Delphi process to analyze social policy implementation: a post hoc case from vocational rehabilitation. Policy Sciences 1993; 26(4): 271-288.

17. Hartdog C, Herbert M. 1985 Opinion survey of MIS managers: key issues. MIS Quarterly 1986; 10(4).

18. Dekleva S, Zupancic J. Key issues in information systems management: a Delphi study in Slovenia. Information & Management 1996; 31(1): 1-11.

19. Chesbrough H. Open innovation: the new imperative for creating and profiting from technology. Boston: Harvard Business School Press, 2003.

20. Chesbrough H. The era of open innovation. MIT Sloan Management Review 2003; 44(3): 35-41.

21. Chesbrough H, Vanhaverbeke W, West J. Open innovation: researching a new paradigm. New York: Oxford University Press, 2006.

22. Bullinger AC, Rass M, Adamczyk S, Moeslein KM, Sohn S. Open innovation in health care: analysis of an open health platform. Health Policy 2012; 105(2): 165-175.

23. McLean S, Protti D, Sheikh A. Telehealthcare for long term conditions. BMJ 2011; 342: d120.

24. Obstfelder A, Engeseth KH, Wynn R. Characteristics of successfully implemented telemedical applications. Implementation Science 2007; 2(25): 1-11.

25. Sood S, Mbarika V, Jugoo S, Dookhy R, Doarn CR, Prakash N, et al. What is telemedicine? A collection of 104 peer-reviewed perspectives and theoretical underpinnings. Telemedicine and e-Health 2007; 13(5): 573-590.

26. Cumbria Rural Health Forum, North West Coast Academic Science Health Network, Academic Health Science Network North East and North Cumbria. Cumbria Strategy for Digital Technologies in Health and Social Care. (Internet). Available: http://www.ruralhealthlink.co.uk/implementing-digital-technology-in-cumbria/strategy (Accessed 31 May 2016).

27. Borkovec TD, Echemendia RJ, Ragusea SA, Ruiz M. The Pennsylvania Practice Research Network and future possibilities for clinically meaningful and scientifically rigorous psychotherapy effectiveness research. Clinical Psychology: Science and Practice 2001; 8(2): 155-167.

28. Gephart RP. Qualitative research and the Academy of Management Journal. Academy of Management Journal 2004; 47(4): 454-462.

29. Cumbria Rural Health Forum, North West Coast Academic Science Health Network, Academic Health Science Network North East and North Cumbria. Digital health and social care: what is being done in Cumbria. (Internet). Available: http://www.ruralhealthlink.co.uk/assets/uploads/reports/Report_Digital_Activities_in_Cumbria.pdf (Accessed 31 May 2016).

30. Mooi JK, Whop LJ, Valery PC, Sabesan SS. Teleoncology for Indigenous patients: the responses of patients and health workers. Australian Journal of Rural Health 2012; 20(5): 265-269.

31. Agrawal A, Bhattacharya J, Baranwal N, Bhatla S, Dube S, Sardana V, et al. Integrating health care delivery and data collection in rural India using a rapidly deployable eHealth center. PLoS Medicine 2013; 10(6): e1001468.

32. Nesbitt TS, Ellis JC, Kuenneth CA. A proposed model for telemedicine to supplement the physician workforce in the USA. Journal of Telemedicine and Telecare 1999; 5(suppl 2): 20-26.

33. Watanabe M, Jennett P, Watson M. The effect of information technology on the physician workforce and health care in isolated communities: the Canadian picture. Journal of Telemedicine and Telecare 1999; 5(suppl 2): 11-19.

34. Hailey D, Jennett P. The need for economic evaluation of telemedicine to evolve: the experience in Alberta, Canada. Telemedicine Journal and e-Health 2004; 10(1): 71-76.

35. Vinals F, Mandujano L, Vargas G, Giuliano A. Prenatal diagnosis of congenital heart disease using four?dimensional spatio-temporal image correlation (STIC telemedicine via an Internet link: a pilot study. Ultrasound in Obstetrics & Gynecology 2005; 25(1): 25-31.

36. Leggett P, Graham L, Steele K, Gilliland A, Stevenson M, O'Reilly D, et al. Telerheumatology - diagnostic accuracy and acceptability to patient, specialist, and general practitioner. British Journal of General Practice 2001; 51(470): 746-748.

37. Czaja SJ, Lee CC, Arana N, Nair SN, Sharit J. Use of a telehealth system by older adults with hypertension. Journal of Telemedicine and Telecare 2014; 20(4): 184-191.

38. Sharma S, Parness IA, Kamenir SA, Ko H, Haddow S, Steinberg LG, et al. Screening fetal echocardiography by telemedicine: efficacy and community acceptance. Journal of the American Society of Echocardiography 2003; 16(3): 202-208.

39. Cottrell E, McMillan K, Chambers R. A cross-sectional survey and service evaluation of simple telehealth in primary care: what do patients think? BMJ Open 2012; 2(6).

40. Chambers R, O'Connell P, Cottrell E. Ensuring the clinical value of telehealth. International Journal of Integrated Care 2013; 13(7).

41. Ward EC, Sharma S, Burns C, Theodoros D, Russell T. The validity of conducting clinical dysphagia assessments for patients with normal to mild cognitive impairment via telerehabilitation. Dysphagia 2011; 26(1).

42. Dunlop S, Coyte PC, McIsaac W. Socio-economic status and the utilisation of physicians' services: results from the Canadian National Population Health Survey. Social Science & Medicine 2000; 51(1): 123-133.

43. Murata C, Kondo K, Hirai H, Ichida Y, Ojima T. Association between depression and socio-economic status among community-dwelling elderly in Japan: the Aichi Gerontological Evaluation Study (AGES). Health Place 2008; 14(3): 406-414.

44. Kanagawa M, Nakata T. Assessment of access to electricity and the socio-economic impacts in rural areas of developing countries. Energy Policy 2008; 36(6): 2016-2029.

45. Beekman AT, Copeland JR, Prince MJ. Review of community prevalence of depression in later life. British Journal of Psychiatry 1999; 174: 307-311.

46. Mumford DB, Saeed K, Ahmad I, Latif S, Mubbashar MH. Stress and psychiatric disorder in rural Punjab. A community survey. British Journal of Psychiatry 1997; 170: 473-478.

47. Nandi DN, Banerjee G, Mukherjee SP, Ghosh A, Nandi PS, Nandi S. Psychiatric morbidity of a rural Indian community. Changes over a 20-year interval. British Journal of Psychiatry 2000; 176: 351-356.

48. Reddy VM, Chandrashekar CR. Prevalence of mental and behavioural disorders in India: a meta-analysis. Indian Journal of Psychiatry 1998; 40(2): 149-157.

49. Tjepkema M. The health of the off-reserve Aboriginal population [Canadian Community Health Survey-2002 Annual Report]. Supplement to Health Reports 2002; 13: 73.

50. Jones S, Tomkinson J. Transformation in practice: an exploration of the large scale implementation of telehealth monitoring in Bristol. International Journal of Integrated Care 2014; 14: 76-77.

51. Taylor DM, Stone SD, Huijbregts MP. Remote participants' experiences with a group-based stroke self-management program using videoconference. Rural and Remote Health Journal 2012; 12: 1947.

52. McPartland S, Hammond L. Application of learning from a local telehealth proof of concept to support dallas delivery. International Journal of Integrated Care 2013; 13: 1-2.

53. Cumbria Rural Health Forum, North West Coast Academic Science Health Network, Academic Health Science Network North East and North Cumbria. Digital health and its application in rural areas: a review of international experience. (Internet). Available: http://www.ruralhealthlink.co.uk/assets/uploads/reports/Report_on_international_learning.pdf (Accessed 31 May 2016).

54. Cooper JB, Newbower RS, Long CD, McPeek B. Preventable anesthesia mishaps: a study of human factors. Anesthesiology 1978; 49(6): 399-406.

55. Zollo SA, Kienzle MG, Henshaw Z, Crist LG, Wakefield DS. Tele-education in a telemedicine environment: implications for rural health care and academic medical centers. Journal of Medical Systems 1999; 23(2): 107-122.

56. Blake H. Staff perceptions of e-learning for teaching delivery in healthcare. Learning in Health and Social Care 2009; 8(3): 223-234.

57. Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of Advanced Nursing 2006; 53(2): 205-212.

58. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing 2000; 32(4): 1008-1015.

59. Hoffman LR. Group problem solving 1. Advances in Experimental Social Psychology 1966; 2: 99.

60. Johnson D. Top ten secrets for a successful workshop. Library Media Connection 2006. (Internet). Available: http://www.doug-johnson.com/dougwri/top-ten-secrets-for-a-successful-workshop.html. (Accessed 15 July 2014).

61. Sbea GP, Guzzo RA. Group effectiveness: what really matters? Sloan Management Review 1987; 28(3): 25.

62. McKenna HP. The Delphi technique: a worthwhile research approach for nursing? Journal of Advanced Nursing 1994; 19(6): 1221-1225.


___________________________________

Appendix I: Outline of Cumbria Rural Health Forum events




Appendix II: Digital health in general practice - general practitioner perceptions




Appendix III: Using technology to manage long term conditions - patient perceptions, attitudes and experiences




Appendix IV: Outline of Twitter chats using the hashtag #CRHFchat (online on https://storify.com/CumbriaRHF/crhfchat)

This PDF has been produced for your convenience. Always refer to the live site https://www.rrh.org.au/journal/article/3738 for the Version of Record.