|Issue 26||Autumn 1999|
Social Research Update is published quarterly by the Department of Sociology, University of Surrey, Guildford GU2 7XH, England. Subscriptions for the hardcopy version are free to researchers with addresses in the UK. Apply by email to email@example.com.
Examining the paradox of achievement gaps
Stephen Gorard is a senior lecturer at the School of Social Sciences, Cardiff University. His research interests include socio-economic segregation in schools, differential attainment by gender, school effects, the role of technology in education, and patterns of lifelong learning. He recently published Keeping a sense of proportion: the politicians error in analysing school outcomes, British Journal of Educational Studies , 1999, 47, 3.
The calculation and discussion of achievement gaps between different sub-groups of students (differential attainment) has become common among policy-makers, the media, and academics. An achievement gap is an index of the difference in an educational indicator (such as an examination pass rate) between two groups (such as males and females). In addition to patterns of differential attainment by gender, recent concern has also been expressed over differences in examination performance by ethnicity, by social class, and by the best and worst performing schools. The concerns expressed in each case derive primarily from growth in these gaps over time.
One of two methods of calculating differential attainment are generally used. The most common uses percentage points as a form of common currency. Thus, if 30% of boys and 40% of girls gain a C grade in Maths GCSE in one year, and 35% of boys and 46% of girls gain the equivalent a year later, the improvement among girls is said to be greater, in the way that six (46-40) is greater than five (35-30). This is justified by its advocates since percentages are, in themselves, proportionate figures. If true, it would mean that girls were now even further ahead of boys than in the previous years. Thus, the gender gap has grown.
The second method calculates the change over time in proportion to the figures that are changing. This approach is advocated by Newbould and Gray in an EOC study of gendered attainment (Arnot et al. 1996, see also Gorard et al. 1999a). For them, an achievement gap is the difference in attainment between boys and girls, divided by the number of boys and girls at that level of attainment. More formally, the entry gap for an assessment is defined as the difference between the entries for girls and boys relative to the total entries.
Entry Gap = (GE-BE)/(GE+BE)where GE = number of girls entered; and BE = number of boys entered.
The achievement gap for each outcome is defined as the difference between the performances of boys and girls, relative to the performance of all entries, minus the entry gap.
Achievement Gap = (GP-BP)/(GP+BP) - Entry Gapwhere GP = the number of girls achieving that grade or better; BP = the number of boys achieving that grade or better.
The interesting point about the two methods is that they give different results from the same data. For example, Gibson and Asthana (1999) claim that the gap in terms of GCSE performance between the top 10% and the bottom 10% of English schools has grown significantly from 1994 to 1998. Their figures are reproduced in Table 1. This shows the proportion of students attaining five or more GCSEs at grade C or above (the official benchmark), for both the best and worst attaining schools in England. It is clear that the top 10% of schools has increased its benchmark by a larger number of percentage points than the bottom 10%. The authors conclude that schools are becoming more socially segregated over time, since within local markets, the evidence is clear that high-performing schools both improve their GCSE performance fastest and draw to themselves the most socially-advantaged pupils (in Budge 1999:3).
This conclusion would be supported by a host of other commentators using the same method (including Robinson and Oppenheim 1998, and Chris Woodhead, in the Times Educational Supplement 12/6/98:5). Similar conclusions using the same method have been drawn about widening gaps between social classes (Bentley 1998), between the attainment of boys and girls (Stephen Byers, in Carvel 1998, Bright 1998, Independent 1998), between the performance of ethnic groups (Gillborn and Gipps 1996), and between the results of children from professional and unemployed families (Drew et al., in Slater et al. 1999).
The second method, using the same figures, might produce a result like Table 2. Although the difference between the deciles grows larger in percentage points over time, this difference grows less quickly than the scores of the deciles themselves. On this analysis, the achievement gaps are getting smaller over time. This finding is confirmed by the figures in the last column showing the relative improvement of the two groups. The rate of improvement for the lowest ranked group is clearly the largest. The bottom decile would, in theory at least, eventually catch up with the top decile (Gorard 1999). The same reanalysis can be done in each of the examples above to show that the gaps between schools, sectors, genders, ethnic groups, and classes are getting smaller over time. This would be the exact opposite in each case to the published conclusions.
To summarise the position so far: using the most popular method of comparing groups over time, there appears to be a crisis in British education. Differences between social groups, in terms of examination results expressed in percentage points, are increasing over time and so education is becoming increasingly polarised by gender, class, ethnicity, and income. However, using the second method, the opposite trend emerges. This is the paradox of achievement gaps. Both methods are used in different studies. Some writers have even used the equivalent of both methods in the same study (e.g. Levacic et al. 1998, Lauder et al. 1999).
Absolute rates are expressed in simple percentage terms, while relative rates (odds ratios) are margin-insensitive in that they remain unaltered by scaling all the numbers in a row or column (as might happen to the class structure over time, for example). In Table 3, 25% of those in the middle class are of working class origin, whereas in Table 4 the equivalent figure is 40% (from Marshall et al. 1997, pp. 199-200). However, this cannot be interpreted as evidence that Society B is more open than Society A, because the percentages do not take into account the differences in class structure between Societies A and B, nor the changes over time (structural differences).
Given a two-by-two table of the form:
From the point of view of social justice... this is of course both crucial and convenient, since our interest lies precisely in determining the comparative chances of mobility and immobility of those born into different social classes - rather than documenting mobility chances as such (Marshall et al. 1997:193 ).The cross-product ratio for Table 3 is 9, and for Table 4 it is also 9. This finding suggests that social mobility is at the same level in each society, despite the differences in class structure between them.
|Destination||Middle class||Working class|
|Destination||Middle class||Working class|
Despite the differences, there are many similarities between the methods and their variants (Darroch 1974). For instance, if 100 girls and 100 boys sit an examination, of whom 30 girls and 20 boys achieve a particular grade, the results produced are as in Table 5 (the cross-product ratio is 1.7, etc.). If in a later test 60 of 100 girls and 40 of 100 boys achieve the same grade, the figures from the first and last methods change, while the others remain the same.
|Method||Test 1 (30%, 20%)||Test 2 (60%, 40%)|
The method of percentage points suggests that the gap between girls and boys has doubled from Test 1 to Test 2, whereas the cross-product ratio suggests that the gap has increased less dramatically. The other two methods suggest no change over time.
It is not always clear (from literature review and personal experience) that the commentators using the percentage point difference approach are aware that there are other, perhaps better, methods. Without restarting the index wars, it would be wise for this issue to be at least debated in relation to the paradox of achievement gaps. If the multiplicative model is preferred, the consequences would be momentous for much existing research, for the cumulated conclusions of some entire fields of endeavour, the validity of many qualitative studies in related areas, for public research funding priorities, and above all for educational policy. As with the earlier debates about class and stratification, other fields of social science investigation would be affected as well.
Arnot, M., David, M. and Weiner, G. (1996) Educational Reform and Gender Equality in Schools, Manchester: Equal Opportunities Commission
Bentley, T. (1998) Learning beyond the classroom, London: Routledge
Bright, M. (1998) The trouble with boys, The Observer, 4/1/98: 13
Budge, D. (1999) Gulf separating weak and strong increases, Times Educational Supplement, 30/4/99: 3
Carvel, J. (1998) Help for boys lagging behind girls at school, The Guardian, 7/1/98:19
Clark, J., Modgil, C. and Modgil, S. (1990) John Goldthorpe: consensus and controversy, London: Falmer
Darroch, J. (1974) Multiplicative and additive interactions in contingency tables, Biometrika, 61, 2, 207-214
Erikson, R. and Goldthorpe, J. (1991) The constant flux: a study of class mobility in industrial societies, Oxford: Clarendon Press
Everitt, B. and Smith, A. (1979) Interactions in contingency tables: a brief discussion of alternative definitions, Psychological Medicine, 9, 581-583
Gibson, A. and Asthana, S. (1999) Schools, markets and equity: access to secondary education in England and Wales, presentation at AERA Annual Conference, Montreal
Gilbert, N. (1981) Modelling society: an introduction to loglinear analysis for social researchers, London: George Allen and Unwin
Gillborn, D. and Gipps, C. (1996) Recent research on the achievements of ethnic minority pupils, London: OFSTED
Goldthorpe, J., Llewellyn, C. and Payne, C. (1987) Social mobility and class structure in modern Britain, Oxford: Clarendon Press
Gorard, S. (1999) Keeping a sense of proportion: the politicians error in analysing school outcomes, British Journal of Educational Studies , 47, 3
Gorard, S. and Fitz, J. (1998) The more things change... the missing impact of marketisation, British Journal of Sociology of Education, 19, 3, 365-376
Gorard, S. and Fitz, J. (1999) Investigating the determinants of segregation between schools, Research Papers in Education , 14, 4
Gorard, S., Salisbury, J. and Rees, G. (1999a) Reappraising the apparent underachievement of boys at school, Gender and Education , 11, 3
Gorard, S., Rees, G. and Fevre, R. (1999b) Families and their participation in learning over time, British Educational Research Journal , 25, 4
Gorard, S., Rees, G. and Fevre, R. (1999c) Two dimensions of time: the changing social context of lifelong learning, Studies in the Education of Adults , 31, 1, 35-48
Independent (1998) Classroom rescue for Britains lost boys, The Independent, 5/1/98: 8
Lauder, H., Hughes, D., Watson, S., Waslander, S., Thrupp, M., Strathdee, R., Simiyu, I., Dupuis, A., McGlinn, J. and Hamlin, J. (1999) Trading in futures: Why markets in education dont work, Buckingham: Open University Press
Levacic, R., Hardman, J. and Woods, P. (1998) Competition as a spur to improvement? Differential improvement in GCSE examination results, presented to International Congress for School Effectiveness and Improvement, Manchester
Lieberson, S. (1981) An asymmetrical approach to segregation, in Peach, C., Robinson, V. and Smith, S. (Eds.), Ethnic segregation in cities, London: Croom Helm
Marshall, G., Swift, A. and Roberts, S. (1997) Against the odds? Social class and social justice in industrial societies, Oxford: Clarendon Press
Robinson, P. and Oppenheim, C. (1998) Social Exclusion Indicators, London: Institute of Public Policy Research
Slater, J., Dean, C. and Brown, R. (1999) Learning gulf divides rich and poor pupils, Times Educational Supplement, 2/4/99:1
Social Research Update is published by:Department of Sociology
Telephone: +44 (0) 1 483 300800
Fax: +44 (0) 1 483 689551
Edited by Nigel Gilbert.
Autumn 1999 © University of Surrey
Permission is granted to reproduce this issue of Social Research Update provided that no charge is made other than for the cost of reproduction and this panel acknowledging copyright is included with all copies.