Outcome evaluation of community health promotion intervention within a donor funded project climate in Papua New Guinea

Introduction: The Australian Agency for International Development (AusAID) funded Women and Children’s Health Project sought to improve the health of women and children throughout Papua New Guinea between 1998 and 2004. The project utilised education, community development and health promotion interventions aimed to increase community support for the health of women and children. Methods: An outcome evaluation in 2006 investigated the long-term impact of the project using a multi-methods approach and covering 10 selectively sampled provinces, 19 districts and 93 communities. Qualitative data were collected from 175 interviews (national to village level) and 77 community discussions. Quantitative data from national, provincial and district levels were examined to attempt to validate findings. Results: The evaluation found new-health-knowledge initiated changes to lifestyle practices and improved physical health and social and economic well-being in villages where volunteers and staff had been trained. Factors influencing success were a healthmotivated person acting as a catalyst for change, empowered leadership through new community governance structures, effective


Introduction
The south Pacific island nation of Papua New Guinea (PNG) covers an area of 462 840 km 2 and has a population of more than 5.2 million 1 .Twenty-three per cent of the population are women of child-bearing age (15-44 years) while 14% are young children (<5 years) 2 .The health system in PNG relies on a network of village aid posts, rural health centres, urban clinics and provincial hospitals 3 .
The Australian Agency for International Development (AusAID) funded the PNG Women and Children's Health (WCH) Project between July 1998 and December 2004 4 .The first author was an external technical adviser to the project.This raises questions of bias, and an attempt was made to address this by reflexivity and continued scrutiny by the second author who supervised the work.
The WCH Project aimed to improve women and children's health though enhanced quality and coverage of rural health services throughout all 19 provinces and the national capital district of PNG 5 .Activities at national, provincial, district and community level were implemented with a substantial budget of AU$32.5 million 6 .Key health-service activities included in-service training for district health workers, and improvements to the cold chain vaccine supply system.
Community activities sought to increase support for the health of women and children through behaviour and attitude changes in relation to sanitation, hygiene and housing.Rural health workers were identified as the target audience with activities focused on strengthening and expanding existing health volunteer programs.
Village health volunteers (VHVs) are health educators in their community.They link the village and health service; encourage a healthier lifestyle; help coordinate community health activities; support prevention of health problems; and encourage the use of maternal and child health services.
Women and men who are respected and chosen by their community and have limited literacy can qualify for training as a VHV.
The link between agencies and personnel involved is shown (Fig1).The changing political and economic environment and foreign aid development context resulted in a 're-scope' of project objectives and outputs mid-way through the project in 2001 7 .
The Ottawa Charter for health promotion and the World Health Organisation's (WHO) 'healthy village settings' approach provided the conceptual basis for the approach taken 8 .The WHO strategies of developing community capacity for self-reliance in health through changes in lifestyle, healthy environment and strong social networks were emphasised 8 .Key indicators of a healthy lifestyle were incorporated into two visual tools for monitoring community and family progress (eg 'healthy community' and 'healthy family').The visual tool used 16 simple line drawings to depict each indicator (Fig2 is an example of one tool).These tools helped the community understand the link between the CAP process (community development principles) and VHV activities (educating about health) and the relationship to WHO's 'healthy village' 9 .

Line of management
This article examines the impact of the CAP and VHV interventions on community support for the health of women and children as a result of the project.

Methods
Two years after completion of formal project activities an outcome evaluation investigated the long-term impact of these community intervention activities on maternal and child health.The evaluation investigated how CAP and VHV program activities influenced interaction between the community and rural health workers.The evaluation also investigated whether the CAP and VHV programs influenced healthy behaviours and increased the use of maternal and child health services.
The evaluation was conducted in PNG between March and September 2006.A multi-methods approach to collecting qualitative and quantitative data was used to assess the interaction between the community and health workers and the community's use of health services 10 .The rapid assessment, response and evaluation (RARE) process was used to collect, synthesise and analyse qualitative data 11 .
Significant stories of community activities and interaction with the health system were collected as a culturally appropriate way of relating experience to knowledge.
The research setting was the PNG communities and rural health workers who were scattered across extremely rugged terrain (Fig3).The evaluation covered 10 of the 20 provinces (50%), all 4 regions, 19 of the 89 districts (21%), 44 rural health facilities (8%) and 93 isolated communities that were recipients of WCH project input 10 .
A multi-level targeted sampling technique was used to identify communities and rural health facilities within sampled provinces and districts (Fig4).One district per province and 2 health facilities within each district were sampled.Two communities in the catchment area of each health facility were sampled.This model was replicated in each province sampled.Selection was based on likely variability of uptake.Communities investigated were along the coast, in the hinterland, up mountain ranges and on outer islands.
Data were collected through interviews, group discussions, site visits, opportunistic observations of behaviours and health record audits.Informal interviews were conducted with 175 stakeholders, from national (n = 18), provincial (n = 34) health officials, district health workers (n = 58) and villagers (n = 3) 10 .Health centre staff, responsible for maternal child health service delivery, health promotion activities and community health initiatives were interviewed together with aid post workers.A total of 77 communitylevel group discussions were conducted 10 .An additional 15 community site visit observations were made where noone was available for interview.Government and nongovernment organisation documents were studied and a health record audit undertaken.The Australian government donor agency, AusAID, financed field work activities.

Committee and Charles Darwin University Human Research
Ethics Committee provided ethical approval for the research.According to WHO's Ottawa Charter, directing action towards changing social, environmental and economic conditions helps individuals 'achieve their fullest health potential'…'by taking control of those things which determine their health' (p.1) 8 .The best way is to judge the intervention's ability to modify risk factors and behaviours, and the personal, social and environmental factors shaping them 12 .A successful health outcome, therefore, can be defined as 'a change in the health of an individual or group which is attributable to an intervention' (p.29) 12 .Green and Kreuter 13 define 'effectiveness' as the proportion of people intended to receive the program who changed as intended.
Demonstrated outcomes for this study were categorised according to the amount of change in the community that could be attributed to WCH project community interventions.The term 'successful' was assigned to a community, based on the PNG healthy islands framework criteria for a 'healthy community' setting 9 .Assessment of each community was according to the degree to which they met the minimum requirements (Table 1).The Healthy Community visual tool (Fig2) was used as a guide by community leaders when reporting to researchers on their community's achievements.Qualitative data from interviews and discussions were analysed according to recurring themes.Quantitative data from health records were compared with qualitative findings.

An overview
The evaluation identified the sustained impact of community intervention activities more than 2 years after formal completion of the project.The CAP and VHV program activities positively influenced interaction between the community and rural health workers.Changes in 'successful' communities demonstrated more healthy behaviours and increased use of maternal and child health services.Discussions with individuals and community leaders identified reduced incidence of illness and improved physical health in families of those villagers who made behavioural and environment changes to improve their health.Previously they believed sanguma (sorcery) caused illness.Increased knowledge and understanding of the 'real' (root) cause of illness motivated behaviour changes as individuals realised that unhealthy practices were making them sick (Fig5).Individuals reported experiencing benefits from simple, small and inexpensive changes (such as improved sanitation and waste disposal practices) that motivated changing traditional ways (Fig6).Positive benefits from these changes have resulted in people accepting a more hygienic environment and improved sanitation practices as the social norm (Fig6).Participants repeatedly told how they now realise improvements are not likely to come from the 'government' and so they have taken up the challenge to improve the environment themselves.Changes in one community have influenced nearby communities because they see the benefits to family health in their neighbours.Examples of changed behaviours and hygiene practices included people using a pit latrine rather than defecating in the bush behind the house, in the garden or on the beach.
Pigs have been removed from areas of human habitation and fenced in an enclosure.Some communities had relocated to allow each family space to build a new house with improved ventilation, a pit latrine and small vegetable garden.One community's resourcefulness provided water for household purposes by digging a drain and allowing nearby stream water to flow alongside the road through the village.Some families built raised platforms as a drying rack for cooking and eating utensils, protecting them from contamination by dirt, flies and animals (Fig6).'Successful' communities often carved out an access road to the village, erected fences around each family's allotment and planted colourful shrubs along the fence line.This appeared to contribute to a sense of community pride, social cohesion and harmony.
Individuals in 'successful' villages made statements such as:   One community had established their own farm raising fingerlings, chicks, rabbits and vegetable seedlings for free distribution to individuals and families.The observed impact in 73 sampled communities are summarised (Table 1).Participants in large community meetings described how improved relationships had reduced social problems caused by using marijuana and alcohol, and smoking.

Reported consequences of change
The role of women in society is changing in PNG, as elsewhere, because increasingly men respect women's opinions.Attitude changes were reported to help community leaders better take up the challenge of responsibility for community health and development.Leaders believed establishing a village health development committee (a strategy advocated through the VHV program) helped strengthen their authority, improved cooperation and respect and increased participation in weekly community activities.
Observations of the numbers of clients seeking curative treatments at aid posts in one district clearly demonstrated the impact of community health education in reducing illness and the health worker's workload.These health workers remarked how this had freed them from the clinical tasks associated with treatment to provide health education and assist communities in promoting healthier lifestyle practices.
Community leaders and local women in 69 villages (84%) surveyed spoke of how women are no longer dying in childbirth.The reasons given were that women, now supported by a VHV, attend antenatal care and are more likely to give birth at a health facility.Although local health worker data confirmed these statements, they could not be verified statistically because the health information system does not disaggregate village-level data.The study found community changes occurred relatively quickly within matrilineal regions because women had more influence on initiating changes.Those within patrilineal regions were restricted by male dominance and appeared to be more reluctant to change from traditional ways.The men needed more convincing of the benefits to society before considering the introduction of change.However, those communities with a strong religious affiliation had a sense of connectedness that was quickly strengthened following the intervention.Leaders acknowledged the need for fresh impetus and support from district and provincial health staff to enhance their motivation to maintain changes in a healthier community.

Reasons for change
Certain characteristics appeared to be contributing to positive effects that were absent in 'less successful' or

Discussion
The principles of health promotion and community development seek to empower individuals to take control of their lives by making decisions that improve health and lifestyle [13][14][15] .Access to reliable information and the role individuals play in influencing their own health through behaviour are recognised as important factors in changing health behaviour 15,16 .An individual's experience of good physical health is intricately linked to their feeling of control over their own life and reinforces maintenance of behaviour change 14,15 .
Evidence from this study suggests new-health-knowledge initiated attitudinal and behaviour changes in personal and environmental hygiene that contributed to social and economic wellbeing.A renewed sense of belonging from reworked community leadership reduced resistance to change and influenced individual participation in initiating and maintaining changes.Increased trust, reciprocity and collective action were evident in this study.This outcome is consistent with theories of community-based development and health promotion.It is also consistent with the philosophy of a 'bottom-up' approach empowering individuals to change the balance of power and increase their self-reliance in health 13,14 .
This research shows families in 'successful' communities now live in a cleaner environment with increased income, better housing and road access.All of these improvements were locally generated.Improved hygiene practices helped reduce the burden of disease.These include improved sanitation and waste disposal practices, with animals fenced away from human habitation and raised dish racks used to protect utensils from contamination.An improved diet and a wider variety of available foods appeared to have influenced the health of children and their learning ability.
A more people-centred approach (one that achieves development 'with', rather than 'for' the people) has frequently demonstrated positive long-term outcomes 13,17 .
Community ownership is basic to development, supporting a sense of community identity and sustainability of changes 14,15 .In this study community members who were engaged in addressing their community needs were enthused to change.The new governance structure generated renewed respect for leadership and a willingness to participate, claim 'ownership' of changes, and commit to future changes.
The link between ultimate vision and immediate outcomes is an important component of community health development programs 14,15 .Studies have demonstrated the link between vision and clear goals, for the immediate outcomes of activities are vital to success in health promotion 15,18,19  The evidence collected for this research from many disparate villages demonstrated those communities, which themselves initiated changes, had maintained those changes more than 2 years after the completion of project input.They explained this was because they owned these changes and were proud of their achievements.This confirms Green & Kreuter's 13 predisposing, reinforcing and enabling process, and the use of a 'bottom-up' approach to community health and development.A partnership approach to working with communities was evident in 80% of sampled districts in 2006.Waisbord 20 examined the impact of a capacity building intervention on Peruvian health worker performance in health promotion that supported improved community health practice.Training alone was found to be insufficient, providing no immediate or tangible effect on health worker performance 20 .This research identified that intensive 'oneoff' training of health workers in the CAP process alone failed to achieve a sustainable outcome.

Figure 1 :
Figure 1: Titles, roles, and relationships of individuals engaged in the Women and Children's Health project community interventions between 1999 and 2003.

Figure 2 :
Figure 2: Sample of the Healthy Community visual tool.

Figure 4 :
Figure 4: Multi-level, targeted sampling technique used to identify communities and rural health facilities within the sampled provinces and districts.

Figure 6 :
Figure 6: Example of a 'successful' village environment, Western Highlands Province.
Evidence of the intervention's success varied among communities, within districts and throughout provinces.A combination of strategies appeared to have maximised effects.These included CAP process training, exposure to a model 'successful' community, the use of visual tools, and additional training for VHV program managers and trainers.Training in the CAP process alone failed to provide the desired outcome.Success in initiating hygiene and environment changes was attributed to the combination of health workers and trained male and female VHVs working together.

'
unsuccessful' communities.Five success factors were identified.The first was a motivated health individual acting as catalyst for change, driven by vision and commitment.In 'successful' communities sampled, this was someone with authority and community respect, usually a health worker, community leader or VHV.Second, based on the existing local government structure, a new governance structure was established in most 'successful' villages, namely the village health and development committee.This committee strengthened the capacity of community leaders and provided impetus for change.Third, those exposed to a 'successful' community under the WCH Project were empowered by this example to initiate sustainable lifestyle changes in their community.Fourth, visual tools depicting healthy family and village settings provided a pictorial guide that raised awareness of the need for change and provided a checklist of progress.Fifth, VHVs acted as a link between their community and local health workers.New knowledge and skills had empowered VHVs to provide education on healthier lifestyle, and motivated community leaders to initiate changes.Volunteers were acting as role models in their community and other individuals followed their example.Volunteers were encouraging families to use maternal and child health services.District level health workers had trained community leaders in the CAP process, promoted healthier practices and were supporting VHVs.This had created an atmosphere of partnership and cooperation between health workers and the community.Evidence of the strength of cooperation was observed during community discussions in 'successful' villages, highlighting the effectiveness of such collaborations in creating healthier communities.Those health workers who understood the relationship between social and environment factors and common illness in society were better equipped to influence community changes.Community members commented on the working partnership between VHVs and local health workers.This evidence of a good relationship was observed to be strongest in communities where local health workers had been actively involved in training VHVs.As a result, leaders from these communities were more willing to work with these health workers and better support their VHVs.The situation was very different for those VHVs in 'less successful' communities because their community did not support them.Local community health workers and nursing officers continued to view their own role as providing only 'curative' care, giving little attention to preventative health care services or promoting a healthier lifestyle.They were not involved in training and monitoring VHVs.The community, therefore, did not recognise their VHVs and failed to support and reward them in traditional ways.Health education appeared to be seen as an unnecessary intrusion into these employed workers' busy clinical schedules.They had stopped giving health promotion messages because they could not see evidence to suggest their message had been heard.Failure of the project CAP intervention to impact on communities appeared to have been influenced by a lack of understanding of community development approaches.This was exacerbated by a 'top-down' approach.The unwillingness of health workers to share information, weak community leadership and opposition in 'less successful' and 'unsuccessful' communities had inhibited any progress.Lack of community cohesiveness and a gender dominant hierarchy was evident in 'unsuccessful' communities.Community discussions and observations suggested if community leadership was weak then individuals showed little respect for it, and there was little mutual cooperation.This was demonstrated by their lack of support for weekly community activities.In communities where politics and jealousy had affected the leadership there was little sustainable evidence of lasting change influencing health and wellbeing.Also, in communities where national level staff, as distinct from local district personnel, had conducted training, community members failed to claim 'ownership' of changes.This study found less than 1% of the 1100 trained in the CAP process alone (ie without VHV training to complement this) had applied their new knowledge and skills.Where evidence did show a positive effect from CAP training only, this was reputed to be the result of additional intensive provincial and district follow-up support.Some health workers appeared reluctant to share information and failed to share their new knowledge of the CAP process or newly developed materials for training and monitoring VHVs.Provincial health officials, in this case, had failed to pass on information and materials to implementers at district and health facility level.The findings indicate those trained in CAP process, in combination with VHV program intervention activities, applied their new knowledge and skills successfully.These people were exposed to a 'successful' community model, use of visual tools to raise awareness and monitor progress and additional training in management and teaching techniques.
This study found additional practical learning strategies increased health worker's application of knowledge and skills.These were visiting a 'successful' community, use of visual tools, and additional training in management and teaching techniques.The mutual enhancement of these activities created a synergy that worked to produce a sustainable outcome.Provincial and district level VHV program managers had additional stimulus and motivation through attendance at regular six-monthly technical meetings, alongside monitoring and support visits from national level.This research confirmed success in community health development requires a combination of activities to meet goals, not just 'one-off' training.Conclusion This article has presented the qualitative findings from an outcome evaluation conducted in PNG in 2006.The findings identified sustained impact of community health interventions (CAP and VHV program initiatives) 2 years after formal completion of WCH Project activities.The interventions resulted in communities taking increased responsibility for their health through healthy lifestyle changes.Communities that partnered with the local health system increased their use of health services to help improve the health of women and children.Certain characteristics appeared to contribute to positive effects which were absent in 'less successful' or 'unsuccessful' communities.These were a health motivated individual acting as catalyst for change driven by vision and commitment, a new local governance structure, visual tools, supportive VHVs and well-informed rural health workers.