Comparative validation of self-report measures of negative attitudes towards Aboriginal Australians and Torres Strait Islanders

Introduction: This study sought to determine the construct validity of two self-report measures of attitudes towards Aboriginal Australians and Torres Strait Islanders against an implicit measure of attitude. Method: Total of 102 volunteer participants completed the three measures in a randomized order. The explicit measures of prejudice towards Aboriginal Australians were the Modern Racism Scale (MRS) and the Attitudes Towards Indigenous Australians Scale (ATIAS). The implicit attitudes measure was an adaptation of the Implicit Association Test (IAT) and utilised simple drawn head-and-shoulder images of Aboriginal Australians and White Australians as the stimuli. Results: Both explicit measures and implicit measure varied in the extent to which negative prejudicial attitudes were held by participants, and the corresponding construct validities were unimpressive. The MRS was significantly correlated with the IAT, (r=.314;p<.05) where the ATIAS was not significantly correlated with IAT scores (r=.12).


Introduction
Racial prejudice is an everyday experience for Aboriginal people in Australia [1][2][3][4][5][6] , with the experience of racism having welldocumented detrimental effects on mental and physical health [7][8][9] .Although healthcare professionals' standards of care and values aim for equity and cultural proficiency, evidence suggests that healthcare professionals are no more exempt from prejudice than the rest of the population, which results in ethnocentric and culturally insensitive healthcare provision [10][11][12][13][14][15] .It is argued that these negative prejudicial attitudes towards ethnic minority groups act as major barriers to Aboriginal Australians and Torres Strait Islanders seeking help from mental health services and psychologists, and may to contribute to ethnic disparities in healthcare provision by influencing medical decision-making [16][17][18] .
Assessment of attitudes poses substantial problems to researchers, where explicit articulation of negative attitudes is frowned upon in some contexts.This poses problems for traditional self-report measures in areas of highly sensitive issues, which have the inherent problems of self-presentation bias and demand characteristic biases.The validity of explicit self-report measures of prejudice and stereotyping may be tested by comparing them with alternative measures of prejudice.To date, two self-report measures of attitudes towards Aboriginal Australians have been reported in the literature.However, neither have been validated against a non-self-report measure.
One approach to establishing the validity of these measures would be to compare participants' self-reported attitudes to an implicit measure of prejudice.Although implicit measures are more timeconsuming and difficult to administer, they can be used to determine the extent of biases present in self-report measures.One such implicit attitudinal measure is the Implicit Association Test (IAT), developed by Greenwald, McGhee, and   Schwartz in 1998 to examine unconscious attitudes using a response-latency measure 19 .In the IAT a subject responds to a series of items that are to be classified into four categoriestypically, two representing a concept discrimination such as flowers versus insects and two representing an attribute discrimination such as pleasant versus unpleasant valence.Subjects are asked to respond rapidly with a right-hand key press to items representing one concept and one attribute (eg insects and pleasant), and with a lefthand key press to items from the remaining two categories (eg flowers and unpleasant).Subjects then perform a second task in which the key assignments for one of the pairs is switched (such that flowers and pleasant share a response, likewise insects and unpleasant).The IAT produces measures derived from latencies of responses to these two tasks.These measures are interpreted in terms of association strengths by assuming that subjects respond more rapidly when the concept and attribute mapped onto the same response are strongly associated (eg flowers and pleasant) than when they are weakly associated (eg insects and pleasant).The assumption is that it should be easier for strongly associated concepts to use the same behavioural response than for weakly associated concepts 19 .
The IAT has been shown to be useful for assessing prejudice and attitudes towards people in different groups 20 .For example McConnell and Leibold found implicit racial attitudes influenced the interaction with a Black confederate such that a participant spoke less (r = .51),smiled less (r =.39), made less task irrelevant conversation (r =.32), made more errors in speech (r = .42)and hesitated more often (r =.35) than if the confederate was White 21 .In comparison, the self-report measures used showed no relationship to the participants behavior.Physicians' implicit racial attitudes as measured by the IAT, were strongly related to their decisions to provide thrombolysis to Black versus White patients with myocardial infarction 22 ; whereas, their self-reported attitudes were not related to treatment decisions.Furthermore, a recent meta analysis has shown the IAT to have greater predictive validity (k=32; n=1,699; r =.24) of subsequent behaviour than self report measures of interracial behavior (l=28; n=1,568; r =12), with the difference also evident for intergroup behavior (r =.21 vs .12) 23.
Therefore, the aim of the proposed research was to assess the construct validity of the Attitude Towards Indigenous Australians Scale (ATIAS) and the Modern Racism Scale (MRS) to assess attitudes towards Indigenous Australians by comparing responses with that of an implicit measure of attitudes, the IAT.

Participants
One hundred and two participants completed the explicit and implicit measures online.Two-thirds were female, 77.5% were students, and 68.6% were Caucasian.Ages ranged from 17 to 61 years, with a mean age of 25 years; and 46.5% of the sample were aged between 17 and 20 years.Breakdown of the other demographic categories for this sample is shown (Table 1) .

Implicit Association Test (IAT):
The version used in the current research used images and words to indicate the concepts of White Australians and Aboriginal Australian.The images used to differentiate between Indigenous Australians and Caucasian groups were simple head-and-shoulder line drawings.All images were generated by one Aboriginal Australian artist.Drawings were used instead of photographs on the advice of three senior Aboriginal Australian researchers from Western Australian in order to reduce the stereotypicality of the images in the current version available at IAT website; to ensure the images were not inherently negative; and to ensure the images would not cause offence to any Aboriginal Australians.Words only were used for the attribute component of the stimuli.These were taken from Race IAT data on the Project Implicit website and were originally selected from pervious normative work 24 .
Although no substantial effects have been found for task order when using both implicit and explicit measures in the same session, randomizing or counterbalancing the presentation order of tasks across participants is still strongly recommended 19 .Therefore, the order of presenting explicit and implicit measures was counterbalanced in the present study.Presentation of the explicit measures was randomized, regardless of whether they were presented before or after the implicit measure.The IAT have also been programmed to be presented in varying orders to avoid the potential for order and practice effects and so for half the time the compatible pairings were presented first (White-pleasant), while for the other half of the time the incompatible prejudicial pairings were presented first (Aboriginal Australian-pleasant).
The scoring algorithm described and tested by Greenwald and colleagues 25,26 was followed to calculate D scores for the IATs, this being the metric that indicates the extent to which the individual's response indicates a positive or negative prejudicial attitude.D values can range between -2 and +2 27 , where negative D values indicate an association inconsistent with the stereotype, in this case a positive prejudicial attitude towards Aboriginal Australians, and positive D values indicate an association consistent with the stereotype, a negative prejudicial attitude towards Aboriginal Australians.
Attitudes Towards Indigenous Australians Scale (ATIAS): This 18 item scale is a measure of attitudes utilizing an explicit questioning format and Likert-type 7 point scaling 28 .Respondents are asked to indicate the extent to which they agree/disagree with statements such as 'Aborigines would be lost without White Australians in today's society'.The scale includes both positively worded items, such as 'Aboriginal people work as hard as anyone else', and negatively worded items.This approach was adopted to reduce the potential for acquiescence bias.A respondent's scores for each item are summed for a total score indicating their attitude towards Indigenous Australians, such that the higher the score the more negative their attitude.The possible range of scores for the ATIAS is 18 to 126, with a midpoint of 63.The reported internal consistency for the scale is an a of 0.93 28 , and was 0.84 in this sample.

Modern Racism Scale (MRS): Originally developed by
McConahay, Hardee and Batts in 1981 29 , the seven-item MRS was adapted for the Australian context by Augoustinos et al in 1994 1 .An example item in the original scale is 'It is easy to understand the anger of Aboriginal people in Australia'.The word 'blacks' from the original scale was replaced with 'Aborigines', with two questions receiving more drastic revision to more accurately reflect the Australian context (eg 'Aborigines are getting too demanding in their push for land rights').Respondents are asked to indicated their personal agreement with each item using a five-point scale ranging from -2 (Disagree strongly) to +2 (Agree strongly).Possible scores range from -14 indicating low or no prejudice, to +14 indicating high prejudice, with zero as the midpoint.The scale has shown good internal consistency, with Cronbach's α ranging from 0.82 to 0.85 1 with the internal consistency in the current study being 0.64.

Demographic variables
As previous studies have indicated that males, people from lower educational backgrounds and older individuals tend to report more negative attitudes toward Indigenous Australians, these data were collected 28 .Ethnicity was recorded to see if positive or negative attitudes towards Indigenous Australians varied according to ethnic background.Finally, participants were also asked to indicate whether they speak English as a first or second language in order to check that language comprehension and understanding was not confounding the results, as the explicit measures were all in English.

Procedure
After obtaining ethics approval (#RA/4/1/4122) from the University of Western Australia Human Research Ethics Committee Research Services, the IAT and self report questionnaires were set up online for access by participants.
A convenience sample consisting of students in the School of Medicine and School of Psychology at UWA, students at Polytechnic West, and Curtin University, Western Australian, as well as student and professional contacts of the researchers, was recruited.Data collected online was regularly examined and those psychology student participants who provided their student number were contacted by email in order to grant experimental credit for their participation.Individuals were provided with an information sheet, an anonymous link to complete the questionnaires and IAT tasks online.In addition an online debrief sheet provided an explanation confirming the purpose of the study, and information about further discussion and clarification (by appointment with the researcher), if required.
Completion and submission of responses to the questionnaires and computer tasks were taken to indicate consent to participate voluntarily in this research.
The presentation of the surveys was randomly ordered for each participant.Reminders to complete the questionnaires were sent to contacts at regular intervals during a two-month data collection period.

Results
In cases without complete data for all explicit and implicit measures, participants' data were removed from analysis in order to eliminate variations in the sample size included in analyses and to ensure the integrity of the results.Number of valid cases, the mean score for the explicit measures, and the mean D value for the IATs, and the standard deviations for each of the measures are provided (Table 1).
In order to test for normality of data distribution, the onesample Kolmogorov-Smirnov test was conducted for the self-report measures, and for the response latencies in the IAT, with all measures evidencing distributions that were not significantly different from normal.
In the current sample ATIAS scores ranged from 25 to 109, with higher scores reflecting higher levels of negative attitudes towards Aboriginal Australians.Two-thirds (67.5%) of the sample scored below the midpoint, thereby reporting a positive attitude towards Indigenous Australians.
In the current sample, scores on the MRS ranged from -11 to +7, with 76% scoring zero or below, indicating they held a positive attitudes towards Aboriginal Australians.In the current sample, D ranged from -1.91 to .74 for the Caucasian and Aboriginal Australian IAT, with the D values in the current sample suggesting that 52% had positive attitudes towards Aboriginal Australians.Of the remaining sample, 1.2% exhibited no preference of association (ie D=0.00), while 47% of participants exhibited a negative prejudicial attitude towards Aboriginal Australians (Table 2).Compared with the IAT, both the ATIAS (χ2=11.45;df=2; p <.01) and MRS (χ2=23.80;df=2; p <.001) identified significantly fewer people having a negative attitudes towards Aboriginal Australians.

Demographic characteristics and attitudes
There was no significant relationship for sex or age and responding to either the explicit or implicit measures.However, there was an effect of education, with participants with a TAFE qualification as their highest level of attainment exhibiting relatively more prejudice on both the ATIAS and MRS (F(3,81)=4.29 and F(3, 81)= 6.57 respectively) than those with higher levels of educational attainment; there was no significant relationship between IAT and education.It was also noted that individuals with English as their second language took significantly longer to respond to items on the ATIAS (t(81)= -2.71 respectively), but there was no evidence of this effect for the MRS or the IAT.

Construct validity of the explicit and implicit measures
The construct validity of the ATIAS and MRS was tested by comparing the scores on the explicit measures with D score for IATs.As the data in Table 3 indicate, the MRS correlated significantly with the D value on the IAT (r =.31: p<.05).
That is, individuals scoring positive on the MRS (and exhibiting more prejudice) also scored more positively on the IAT.However, the ATIAS did not correlate significantly (r =.12) with the D score on the IAT.
In light of the significant correlations between some of the demographic variables and the MRS and ATIAS, the correlations were repeated, using a partial correlation to control for education and English language capability.The relationship between the ATIAS remained non-significant, and the relationship between the MRS and IAT remained significant although this decreased slightly (r =.29; p <.05) .

Discussion
The results of the present study support previous research conducted with the IAT and explicit measures of racial prejudice [20][21][22][23] , in that the IAT suggested more people held a negative attitude towards Aboriginal Australians than would be indicated by their responses to the self-report measures.
Only 33% of the sample reported prejudiced attitudes on the ATIAS, and 24% on the MRS indicated they held prejudiced attitudes, compared with 48% identified as holding negative prejudiced attitudes towards Indigenous Australians by the IAT.The ATIAS did not correlate with the IAT and possibly requires greater English language capability than either or the MRS or the IAT.The fact that that the ATIAS indentified more people as prejudiced than the MRS, and yet did not correlate with IAT as the MRS does, requires further study.
Despite underestimating prejudice in the sample in comparison with the IAT estimate, scores on the MRS were positively correlated with responses in the Caucasian and Indigenous Australian IAT.It is also of note that the MRS was not influenced by English as a second language when participants responded.This suggests that using the MRS to measure explicit attitudes towards Indigenous Australians may be more appropriate than the ATIAS.

Criticisms of the IAT
The IAT has been proposed as a method for assessing prejudice and other attitudes that circumvents issues of socially desirable responding and demand characteristic biases 29 .Critics of the IAT have expressed concern about the veracity of this contention.Four criticisms are particularly pertinent to measuring racial attitudes: (i) a cognitive skill confound; (ii) the influence of extra-personal associations; (iii) public versus private administration differences; and (iv) the 'fakeability' of the IAT.It should be noted that IAT, although able to identify a negative prejudice, does not identify the content of that prejudice, which needs to be accessed explicit self-report assessments.McFarland and Crouch identified and demonstrated a cognitive skill confound inherent in the IAT methodology 30 .
The IAT measures response speeds to categories and the degree to which these are slower when the categories are incongruent to the individual compared to when they are congruent.Therefore, a general cognitive inability to suppress incongruence might influence response speeds on the IAT 30 .To address this problem we followed the recommendation to use both positive and negative exemplars in the IAT, mirroring the methodology of DeHouwer 31 .Further, we followed the recommendation to increase the number of practice trials to 40 in block 5 when the pairings of categories and attributes switch, which has been shown to reduce the influence of order effects 18 .Together these strategies will counter the effect that a cognitive skill confound will have on the IAT.
The second critique of the IAT suggests that prior attitudeirrelevant information in memory may affect response speeds to the task 31 .Although these extrapersonal associations may be problematic in some IATs, the influence of society's normative attitudes towards Indigenous Australians is valuable in disentangling racial attitudes in Australia.Therefore, even if there is contamination occurring in the current study, this contamination is likely to reflect how Australian society has conditioned individuals to respond to the issue of Indigenous relations, and this is what the tools attempt to measure.
A third criticism of the IAT is questioning the assertion that it is resistant to socially desirable patterns of responding.One model of attitudes suggests that automatically activated attitudes can be overridden if an individual has the motivation and opportunity to override the effects of the spontaneous attitudes 21 .Clearly in the present research there was little congruence between the explicit and implicit attitude scores, which suggests that there may have been motivation to alter responses on the ATIAS and MRS to appear less prejudiced.This lends support to the suggestion that the IAT is less susceptible to socially desirable responding.To address the argument that situational and contextual biases might affect responding, the authors enabled a private administration of the IAT by hosting it online, and used non-typical pictograms as stimuli rather than photographs of stereotypical Aboriginal Australians.
Fourth, some question the IAT's resistance to faking.However, Kim found that individuals did not spontaneously work out how to mask their attitudes and respond more favourably 32 .When instructions on faking were given, participants were only partly successful in producing faked responses because they were unable to speed up responses to the Black and pleasant combination 32 , results replicated with a personality IAT 33 .This suggests that the IAT elicits automatically activated attitudes to target categories, and unless there is incentive and instructions on how to consciously attenuate these attitudes, these attitudes will not be subjected to deliberate attempts to respond differently.No incentive was given to participants in the current study, so it is unlikely that deliberate attempts were made to respond differently on the IAT.
Despite these criticisms, there is still evidence that the IAT is a better predictor of subsequent behaviour and decisions than explicit measures [21][22][23] .Thus it would seem to be an appropriate standard by which to compare self-report measures.

Conclusion
Almost 50% of a young, well-educated population, that included non-Aboriginal Australian ethnic minority groups, exhibited prejudice towards Indigenous Australians when measured by the IAT.The extent of prejudice recorded is concerning and flags an opportunity to invest time in addressing these issues in undergraduate curricula.Furthermore, when using self-report measures of attitudes towards Aboriginal Australians, the current data suggests that the MRS has better construct validity than the ATIAS and is not affected by English proficiency.

Table 1 :
Summary of the demographic variables and descriptive of measures ATIAS, Attitudes Towards Indigenous Australians Scale; HSC, higher school certificate; IAT D, Implicit Association Test; MRS, Modern Racism Scale.

Table 2 :
Summary comparison of percentage of sample that was low, neutral, or high on prejudice towards Indigenous Australians

Table 3 :
Correlations between measures