How good are routinely collected primary healthcare data for evaluating the effectiveness of health service provision in a remote Aboriginal community ?

Introduction: Evaluation and monitoring of primary health care requires the establishment and maintenance of an appropriate data system. This study reviews the application and effectiveness of the Communicare data management system in the delivery of health services to the Fitzroy Valley in the Kimberley region of Western Australia. Methods: Key demographic fields (sex, date of birth and Aboriginal status) were examined for completeness (whether the date fields were all completed and correct when compared with the paper file) while the ‘conditions’ field was examined for accuracy. Three chronic diseases (diabetes, hypertension and chronic kidney disease) in adults and age-specific incidence for four acute diseases (otitis media, gastroenteritis, lower respiratory tract infection and skin infection) in children were included. Results: Completeness of chosen demographic fields was 100% for date of birth and sex and 98% for Aboriginal status. Chronic conditions matched the paper files 100%, while the recording of acute conditions was incomplete. Among older adults (≥55 years) the prevalences of diabetes, chronic kidney disease and hypertension were 43%, 42% and 39% respectively. Age-specific incidence of acute conditions was highest in the 0–4 years age group where 25% had had at least one episode of otitis media and 20% at least one episode of skin infection. Conclusions: The recording of demographic and chronic disease data was complete, but lower for acute conditions. Routinely collected data have a number of limitations, but nonetheless are a feasible way to establish population health indices, particularly for


Introduction
An effective primary health service is one that improves health outcomes and sustains the optimal health of its population 1 .Monitoring health service effectiveness requires a comprehensive data collection system that includes required personal and diagnostic data.These data are essential for two key reasons.First, knowing the health needs of a population enables health services to plan and target their services to maximise health outcomes.Second, collecting information about disease rates at regular intervals completes the continuous quality improvement cycle necessary to determine health service performance.
Aboriginal Australians living in remote areas have worse health outcomes compared to non-Aboriginal Australians, both nationally and in rural and remote areas 2,3 .It is therefore particularly important for health organisations serving remote Aboriginal Australians to determine and monitor the prevalence of important diseases in the population in order to ensure that health resources are being directed for greatest benefit.Such disease monitoring systems need to be easily accessible and not require large investments of staff time or money, particularly given the high turnover of staff in remote health centres, which requires frequent training to be provided.Data routinely collected by computerised clinical management systems are both accessible and available; however, as with all secondary data, there can be issues with data quality.The findings of the present study are similar to others: routinely collected electronic health is adequate for health service monitoring, auditing and evaluating service provision for continuous quality improvement, despite not being 100% accurate as required for research purposes 4,5 .
This study seeks to determine whether data in the existing clinical management system Communicare (Communicare Systems Pty Ltd; http://www.communicaresystems.com.au) is of sufficient quality for estimating disease burden, (to identify areas in which data entry and hence data quality can be improved and to calculate baseline rates of important diseases in a remote Aboriginal population, which can be used to inform health service program planning and evaluation.

Background
The Fitzroy Valley is a remote area of Western Australia's Kimberley region, located approximately 400 km from the nearest regional centre (Broome) and 2500 km from the state capital (Perth).The Fitzroy Valley population is estimated at about 4500 (although there are known challenges to determining an accurate resident population due to high population mobility), of whom the majority are Aboriginal 6,7 .Approximately 1500 people live in the main town of Fitzroy Crossing, with the remainder living in more than 45 small communities of no more than 250 residents located throughout the surrounding valley 6,7 .The majority of clinical services in the Fitzroy Valley are provided either by the Fitzroy Valley Health Service, which provides both primary and tertiary care from the hospital situated in Fitzroy Crossing, or the Kimberley Population Health Unit, which provides primary health care via outreach clinics in the larger communities.Nindilingarri Cultural Health Services, a community controlled organisation, provides non-clinical health services for the Fitzroy Valley, including cultural safety, environmental health service, aged care and health promotion.
Over the past 5 years a formal partnership between all health services reoriented the focus of health service provision towards an increasingly integrated and holistic primary health care model 8 .This transition included the introduction of the clinical management system Communicare, designed to improve patient management and the sharing of information between the health services in the Fitzroy Valley.From 2005, a gradual introduction of Communicare took place with transfer of patient health details from paper records to the electronic system.This included both administrative efforts to transfer previously diagnosed conditions and recent medical encounters (from 2000 onwards) as well as updates by clinical staff of medical records with relevant history when patients were reviewed.From the beginning of 2010, Communicare has been the primary patient record system used in the Fitzroy Valley.Its introduction has provided the potential to use Communicare to monitor the burden of disease and health outcomes as part of an ongoing health service evaluation framework.

Methods
Data were extracted using the report function of Communicare.Demographics, illness condition and the date of condition recorded between the 1 January 2000 and 19 June 2011 were obtained for all patients presenting at any of the clinical services with a current address in the Fitzroy Valley.Data quality was determined in two domains: completeness of demographic fields and accuracy of the 'conditions' field.(The conditions field is non-compulsory and allows clinicians to enter a diagnosis from an International Classification for Primary Care-2 plus 'picking list', for each consultation.It is used to generate populationbased reports or audits of various conditions.)For completeness, the demographic fields chosen for examination were Aboriginal status, date of birth and sex, with completeness defined as any value entered in the specified field other than 'not stated' or 'not applicable'.Accuracy of the conditions field was determined as follows: five conditions (two for those aged <15 years and three for those aged ≥15 years) were chosen based on their high prevalence and health implications for remote Aboriginal populations (Table 1).For each of the five conditions, a list of records was generated that did not have the chosen condition recorded in the conditions field.From each of these lists, 50 records were randomly selected and carefully examined to determine whether the condition was listed elsewhere in the patient record (eg in the free text).See Figure 1 for an example of how this method was used to determine accuracy of records for diabetes mellitus.
Rates of disease were calculated for the five conditions examined for accuracy as well as for two additional conditions for those aged <15 years.Prevalence by age group was calculated for chronic conditions and age-group specific incidence for acute conditions.The calculation of disease rates was restricted to the Aboriginal population only, and given recognised difficulties with determining accurate population figures for the Fitzroy Valley 7 the Communicare population list was used as the denominator for calculations of disease occurrence.The year in which chronic conditions were first recorded was calculated by extracting the year that the condition was first recorded in the conditions field.Data analysis was conducted using Stata v10 (StataCorp; http://wwwstata.com)and figures created using Microsoft Excel.

Ethics approval
Ethics approval (2011:26) was granted by the Kimberley Aboriginal Health Planning Forum Research Subcommittee, the Western Australian Aboriginal Ethics Council and the Western Australian Country Health Service Ethics Committee.

Data quality
Table 2 shows the quality of the data set.Of the 5227 individual Communicare records for the Fitzroy Valley, completeness of the demographics fields was high.Accuracy of the 'conditions' field by specific condition varied by diagnosis (Table 3).

Disease prevalence and incidence
Disease prevalence and incidence were assessed from data for 3495 Aboriginal residents from 22 communities recorded as current Fitzroy Valley patients in Communicare.The population had a male-to-female ratio of 100:101, and was generally young with 33% aged <15 years, 53% aged 15-49 years and 13% aged ≥50 years.Fitzroy Crossing had the largest population (23%); all other communities had fewer than 15% of the total population (range 0.5-13%).
Figure 2 shows specific incidence, by age group, in children aged <15 years.Otitis media was the most commonly recorded condition for persons aged <5 years, while skin infections were the most commonly recorded conditions in the 5-9 and 10-14 years age groups.
Figure 3 shows prevalence of important conditions of adulthood.Diabetes was the most commonly recorded condition with an overall prevalence of 17% in those aged >15 years, while renal disease was the second most commonly reported condition with an overall prevalence of 14%.For all chronic conditions, prevalence increased with age, and for those aged ≥55 years, 66% had at least one of the three chronic conditions recorded.The year in which the condition was first recorded is shown in Figure 4, which indicates an increase in conditions recorded until 2006-2007 and then relatively steady numbers from 2008 onwards.

Discussion
This descriptive community study sought to examine the quality of routinely collected clinical data as the basis for calculating rates of important diseases in the Fitzroy Valley population for health service planning and evaluation, and for providing useful feedback to the community through the health service partnership.
Invariably, there are recognised issues associated with using routinely collected primary health care data for population health planning purposes, including lack of completeness and standardisation in clinical coding 4,9,10 .Different data fields in the Communicare system showed some variation in quality.Demographics were well recorded, as were chronic conditions in the conditions field.Acute conditions, however, were less well recorded.In reality, disease rates generated using these data reflect clinician diagnosis, and, in the absence of using validated case definitions, there was no standardised way of recording certain conditions (eg there were 12 different codes that could be used to record chronic kidney disease).Moreover, new diagnoses of chronic diseases (Fig3) are likely to reflect the increasing uptake of Communicare, so the first time a patient with such a condition was seen since the health provider had started using Communicare would be entered as date of onset, rather than the first date the condition was diagnosed.Additionally, this single database lacks power for research that larger sources of primary health care data (eg practice networks) can generate.These issues notwithstanding, routinely collected data entered in Communicare are free, easily obtainable and available to local staff, who can use them to understand the specific health needs of their population.A particular strength of these data is undoubtedly completeness of population coverage, since all healthcare providers in the Fitzroy Valley use Communicare thereby enabling a comprehensive account of the health of this remote community to be recorded.Possibly, acute diseases were not as well recorded because this field is used to create a summary of patient health status.Therefore, while busy clinicians may see filling it in as important when a condition requires ongoing management (such as diabetes), this may not be the case for an acute and/or self-limiting conditions, which clinicians may find easier to record in free text.However, until the capacity to extract data from free text via routine reporting tools is available, it will be necessary to rely on clinicians to enter all coded data accurately for each consultation.
As a result of this study, the health service has initiated several quality improvement strategies to further improve these routinely collected data.For example, a list of standardised recording codes, including clear and easily understandable case definitions, has been finalised for a range of important conditions with regular auditing and data cleaning.A clinician engagement strategy has also been developed that emphasises the importance of the clinician's role in accurately entering data, which will ultimately be used to improve health outcomes.Implementation of these measures will undoubtedly serve to improve data quality for its use in evaluating health outcomes and the targeting of health services.
In terms of calculating baselines rates of diseases, the Communicare data show a very high prevalence of the chronic conditions considered, consistent with other studies of similar populations 11,12 .These rates highlight the importance of the current realignment of health services towards a comprehensive primary healthcare approach to prevent and treat chronic disease.It should be noted, however, that using prevalence rates of chronic diseases over time as the sole evaluation tool for health service improvement would not be sufficient, because of the long lag times associated with development of these diseases, the effect of other confounding factors that influence disease occurrence and possible improvements in screening for chronic disease under the new health service model 13 .Disease prevalence will therefore need to be used in concert with process indicators that are more sensitive to changes in health service provision and will therefore form part of a more comprehensive evaluation framework.Early evaluation using Communicare has already shown how health service provision can be improved 14 .
Although the rates of acute conditions in children are high, they are lower than those found by other studies of similar populations 15,16 .They are likely to be underestimates for two main reasons.The first is under-recording due to not filling in the conditions field.The second is that not all people with these diseases present for clinical treatment.Therefore, using changes in the rate of acute disease to evaluate the new model of health service will need to take these factors into account.
A more useful and sensitive measure for monitoring purposes may be the number of infections per child within a specified time period.The possibility of having this as a 'routine' report is being discussed with Communicare developers.

Conclusions
Good medical information systems are essential for the planning and evaluation of the effectiveness of primary healthcare service delivery.This study found that Communicare is generally an effective tool for determining the burden of disease in a remote population, and has been integrated into an evaluation framework for monitoring health outcomes over time 17 .Although primary healthcare data routinely collected by health authorities invariably have some inherent limitations, they are nevertheless able to be used to establish population health indices with minimal cost and effort in a feasible way.

Figure 1 :
Figure 1: Method of determining accuracy of conditions field -example of diabetes mellitus.

Figure 2 :
Figure 2: Age-group specific incidence of acute conditions in those aged <15 years.

Figure 3 :
Figure 3: Prevalence of important conditions in those aged ≥15 years.

Figure 4 :
Figure 4: Year of first recording for the conditions of hypertension, kidney disease and diabetes.

Table 1 :
Conditions of importance analysed, by age group † Not examined for accuracy

Table 2 :
Completeness of key demographic fields

Table 3 :
Accuracy of conditions field (50 records where disease not included in 'conditions field' examined for each disease)