Social Research update logo
Issue 26 Autumn 1999

Social Research Update is published quarterly by the Department of Sociology, University of Surrey, Guildford GU2 7XH, England. Subscriptions for the hardcopy version are free to researchers with addresses in the UK. Apply by email to

Examining the paradox of achievement gaps

Stephen Gorard

Stephen Gorard is a senior lecturer at the School of Social Sciences, Cardiff University. His research interests include socio-economic segregation in schools, differential attainment by gender, school effects, the role of technology in education, and patterns of lifelong learning. He recently published ‘Keeping a sense of proportion: the “politician’s error” in analysing school outcomes’, British Journal of Educational Studies , 1999, 47, 3.

The calculation and discussion of achievement gaps between different sub-groups of students (‘differential attainment’) has become common among policy-makers, the media, and academics. An ‘achievement gap’ is an index of the difference in an educational indicator (such as an examination pass rate) between two groups (such as males and females). In addition to patterns of differential attainment by gender, recent concern has also been expressed over differences in examination performance by ethnicity, by social class, and by the ‘best’ and ‘worst’ performing schools. The concerns expressed in each case derive primarily from growth in these gaps over time.

One of two methods of calculating differential attainment are generally used. The most common uses percentage points as a form of ‘common currency’. Thus, if 30% of boys and 40% of girls gain a C grade in Maths GCSE in one year, and 35% of boys and 46% of girls gain the equivalent a year later, the improvement among girls is said to be greater, in the way that six (46-40) is greater than five (35-30). This is justified by its advocates since percentages are, in themselves, proportionate figures. If true, it would mean that girls were now even further ‘ahead’ of boys than in the previous years. Thus, the gender gap has grown.

The second method calculates the change over time in proportion to the figures that are changing. This approach is advocated by Newbould and Gray in an EOC study of gendered attainment (Arnot et al. 1996, see also Gorard et al. 1999a). For them, an achievement gap is the difference in attainment between boys and girls, divided by the number of boys and girls at that level of attainment. More formally, the entry gap for an assessment is defined as the difference between the entries for girls and boys relative to the total entries.

Entry Gap = (GE-BE)/(GE+BE)

where GE = number of girls entered; and BE = number of boys entered.

The achievement gap for each outcome is defined as the difference between the performances of boys and girls, relative to the performance of all entries, minus the entry gap.

Achievement Gap = (GP-BP)/(GP+BP) - Entry Gap

where GP = the number of girls achieving that grade or better; BP = the number of boys achieving that grade or better.

The interesting point about the two methods is that they give different results from the same data. For example, Gibson and Asthana (1999) claim that the gap in terms of GCSE performance between the top 10% and the bottom 10% of English schools has grown significantly from 1994 to 1998. Their figures are reproduced in Table 1. This shows the proportion of students attaining five or more GCSEs at grade C or above (the official benchmark), for both the best and worst attaining schools in England. It is clear that the top 10% of schools has increased its benchmark by a larger number of percentage points than the bottom 10%. The authors conclude that schools are becoming more socially segregated over time, since ‘within local markets, the evidence is clear that high-performing schools both improve their GCSE performance fastest and draw to themselves the most socially-advantaged pupils’ (in Budge 1999:3).

Table 1: Changes in GCSE benchmark by decile
Decile 1994 1998 Gain 1994-8
Top 65.0% 71.0% 6.0
Bottom 10.6% 13.1% 2.5

This conclusion would be supported by a host of other commentators using the same method (including Robinson and Oppenheim 1998, and Chris Woodhead, in the Times Educational Supplement 12/6/98:5). Similar conclusions using the same method have been drawn about widening gaps between social classes (Bentley 1998), between the attainment of boys and girls (Stephen Byers, in Carvel 1998, Bright 1998, Independent 1998), between the performance of ethnic groups (Gillborn and Gipps 1996), and between the results of children from professional and unemployed families (Drew et al., in Slater et al. 1999).

The second method, using the same figures, might produce a result like Table 2. Although the difference between the deciles grows larger in percentage points over time, this difference grows less quickly than the scores of the deciles themselves. On this analysis, the achievement gaps are getting smaller over time. This finding is confirmed by the figures in the last column showing the relative improvement of the two groups. The rate of improvement for the lowest ranked group is clearly the largest. The bottom decile would, in theory at least, eventually catch up with the top decile (Gorard 1999). The same reanalysis can be done in each of the examples above to show that the gaps between schools, sectors, genders, ethnic groups, and classes are getting smaller over time. This would be the exact opposite in each case to the published conclusions.

Table 2: Changes in GCSE achievement gaps by decile
Decile 1994 1998 Ratio 98/94
Top 65.0% 71.0% 1.09
Bottom 10.6% 13.1% 1.24
Achievement gap 72.0% 68.8%

To summarise the position so far: using the most popular method of comparing groups over time, there appears to be a crisis in British education. Differences between social groups, in terms of examination results expressed in percentage points, are increasing over time and so education is becoming increasingly polarised by gender, class, ethnicity, and income. However, using the second method, the opposite trend emerges. This is the ‘paradox of achievement gaps’. Both methods are used in different studies. Some writers have even used the equivalent of both methods in the same study (e.g. Levacic et al. 1998, Lauder et al. 1999).

The ‘index wars’

Very similar analyses also occur in social science more generally, and similar problems have arisen in health research (Everitt and Smith 1979), in studies of socio-economic stratification and urban geography (Lieberson 1981), in social mobility work (Erikson and Goldthorpe 1991), and in predictions of educational pathways (Gorard et al. 1999b). Results are disputed when an alternative method of analysis produces contradictory findings. Some of these debates are still unresolved, dating back to what Lieberson (1981) calls the ‘index wars’ of the 1940s and 1950s. In each case the major dispute is between findings obtained using absolute rates (‘additive’ models) and those using relative rates (‘multiplicative’ models).

Absolute rates are expressed in simple percentage terms, while relative rates (odds ratios) are margin-insensitive in that they remain unaltered by scaling all the numbers in a row or column (as might happen to the class structure over time, for example). In Table 3, 25% of those in the middle class are of working class origin, whereas in Table 4 the equivalent figure is 40% (from Marshall et al. 1997, pp. 199-200). However, this cannot be interpreted as evidence that Society B is more open than Society A, because the percentages do not take into account the differences in class structure between Societies A and B, nor the changes over time (‘structural differences’).

Given a two-by-two table of the form:

a b

c d

relative rates are calculated as odds or cross-product ratios [(a/c)/(b/d), equivalent to ac/bd] or disparity ratios [a/(a+c)/b/(b+d)]. Disparity ratios are identical to the segregation ratios used by Gorard and Fitz (1998, 1999). Odds ratios estimate comparative mobility changes regardless of changes in the relative size of classes, and have the practical advantage of being easier to use with loglinear analysis (Gilbert 1981, Goldthorpe et al. 1987, Gorard et al. 1999c).
From the point of view of social justice... this is of course both crucial and convenient, since our interest lies precisely in determining the comparative chances of mobility and immobility of those born into different social classes - rather than documenting mobility chances as such (Marshall et al. 1997:193 ).
The cross-product ratio for Table 3 is 9, and for Table 4 it is also 9. This finding suggests that social mobility is at the same level in each society, despite the differences in class structure between them.

Table 3: Social mobility in Society A
Destination Middle class Working class
Middle class 750 250
Working class 250 750

Table 4: Social mobility in Society B
DestinationMiddle classWorking class
Middle class 750 250
Working class 500 1500
Some previous work has confounded changes in social mobility with changes in the class structure. Nevertheless, disagreement about the significance of absolute and relative mobility rates continues (e.g. Clark et al. 1990, pp. 277-302). Gilbert (1981) concluded that ‘one difficulty with having these two alternative methods of analysis is that they can give very different, and sometimes contradictory results’ (p.119). The similarities to the issue concerning achievement gaps are obvious. In each case, different commentators use the same figures to arrive at different conclusions. One group is using additive and the other is using multiplicative models.

Comparing indices

Four alternative methods have been mentioned for assessing relationships in a simple two-by-two contingency table. The cross-product (or odds) ratio is commonly used to estimate social mobility, and the segregation (or disparity) ratio (or dissimilarity index) can be used for the same purpose, but is perhaps more generally applicable to the analysis of changes in stratification over time. The achievement gap is used to analyse differential attainment by sub-groups, but is also useful for defining differential access to public services. These three methods are all multiplicative. Percentage points differences have also been used in all of these areas as a rough and ready guide which is easy to calculate. This method is additive in nature.

Despite the differences, there are many similarities between the methods and their variants (Darroch 1974). For instance, if 100 girls and 100 boys sit an examination, of whom 30 girls and 20 boys achieve a particular grade, the results produced are as in Table 5 (the cross-product ratio is 1.7, etc.). If in a later test 60 of 100 girls and 40 of 100 boys achieve the same grade, the figures from the first and last methods change, while the others remain the same.

Table 5: Comparing indices across two related tables
MethodTest 1 (30%, 20%)Test 2 (60%, 40%)
Cross-product 1.7 2.3
Segregation girls 1.2 1.2
Segregation boys 0.8 0.8
Achievement gap 0.2 0.2
Percentage points 10.0 20.0

The method of percentage points suggests that the gap between girls and boys has doubled from Test 1 to Test 2, whereas the cross-product ratio suggests that the gap has increased less dramatically. The other two methods suggest no change over time.

Resolving the paradox of achievement gaps?

Although arguments can and have been made for using either multiplicative or additive methods as measures of association in one table, the chief problem lies in their different results when comparing the patterns in two or more tables. Alone among the methods, using percentage points does not take into account the proportion (a+b)/(c+d). Using this method, a commentator can have no genuine idea of the significance of the resulting points difference. For example, if 1% of men were MPs but 0% of women were, this would be an enormous difference and one that social science commentators would be right to draw attention to. On the other hand, if 75% of men and 76% of women were in paid employment, the difference may be of little account. However, both examples yield a score of 1 point using the additive method, suggesting that this method is fine for a rough guide to the presence or absence of a pattern, but of little value as a measure of achievement gaps.


At present, the situation is that the specific method of calculation used to assess changes in relative performance over time determines the result obtained. A consensus about the two methods needs to be reached quickly by the research community, even if it is an agreement to differ. In terms of social mobility research the preference of most commentators on methodology is clear, if less than whole-hearted. Darroch (1974) says ‘on balance, the author believes that Hm [the multiplicative definition] is preferable to Ha [the additive definition]’ (p.213). Gilbert (1981) suggests that the percentage point difference method can be used ‘to assess the association in a percentaged table quickly and roughly’ (p.119), but states an overall preference for the relative ratio methods for the kind of practical reasons described above. Ironically, Marshall et al. (1997) are more dogmatic in their preference for the relative approach for social mobility studies, but include in their own work a percentage point difference approach to relative changes in educational qualifications over time (p.113).

It is not always clear (from literature review and personal experience) that the commentators using the percentage point difference approach are aware that there are other, perhaps better, methods. Without restarting the index wars, it would be wise for this issue to be at least debated in relation to the paradox of achievement gaps. If the multiplicative model is preferred, the consequences would be momentous for much existing research, for the cumulated conclusions of some entire fields of endeavour, the validity of many ‘qualitative’ studies in related areas, for public research funding priorities, and above all for educational policy. As with the earlier debates about class and stratification, other fields of social science investigation would be affected as well.


Arnot, M., David, M. and Weiner, G. (1996) Educational Reform and Gender Equality in Schools, Manchester: Equal Opportunities Commission

Bentley, T. (1998) Learning beyond the classroom, London: Routledge

Bright, M. (1998) The trouble with boys, The Observer, 4/1/98: 13

Budge, D. (1999) Gulf separating weak and strong increases, Times Educational Supplement, 30/4/99: 3

Carvel, J. (1998) Help for boys lagging behind girls at school, The Guardian, 7/1/98:19

Clark, J., Modgil, C. and Modgil, S. (1990) John Goldthorpe: consensus and controversy, London: Falmer

Darroch, J. (1974) Multiplicative and additive interactions in contingency tables, Biometrika, 61, 2, 207-214

Erikson, R. and Goldthorpe, J. (1991) The constant flux: a study of class mobility in industrial societies, Oxford: Clarendon Press

Everitt, B. and Smith, A. (1979) Interactions in contingency tables: a brief discussion of alternative definitions, Psychological Medicine, 9, 581-583

Gibson, A. and Asthana, S. (1999) Schools, markets and equity: access to secondary education in England and Wales, presentation at AERA Annual Conference, Montreal

Gilbert, N. (1981) Modelling society: an introduction to loglinear analysis for social researchers, London: George Allen and Unwin

Gillborn, D. and Gipps, C. (1996) Recent research on the achievements of ethnic minority pupils, London: OFSTED

Goldthorpe, J., Llewellyn, C. and Payne, C. (1987) Social mobility and class structure in modern Britain, Oxford: Clarendon Press

Gorard, S. (1999) Keeping a sense of proportion: the “politician’s error” in analysing school outcomes, British Journal of Educational Studies , 47, 3

Gorard, S. and Fitz, J. (1998) The more things change... the missing impact of marketisation, British Journal of Sociology of Education, 19, 3, 365-376

Gorard, S. and Fitz, J. (1999) Investigating the determinants of segregation between schools, Research Papers in Education , 14, 4

Gorard, S., Salisbury, J. and Rees, G. (1999a) Reappraising the apparent underachievement of boys at school, Gender and Education , 11, 3

Gorard, S., Rees, G. and Fevre, R. (1999b) Families and their participation in learning over time, British Educational Research Journal , 25, 4

Gorard, S., Rees, G. and Fevre, R. (1999c) Two dimensions of time: the changing social context of lifelong learning, Studies in the Education of Adults , 31, 1, 35-48

Independent (1998) Classroom rescue for Britain’s lost boys, The Independent, 5/1/98: 8

Lauder, H., Hughes, D., Watson, S., Waslander, S., Thrupp, M., Strathdee, R., Simiyu, I., Dupuis, A., McGlinn, J. and Hamlin, J. (1999) Trading in futures: Why markets in education don’t work, Buckingham: Open University Press

Levacic, R., Hardman, J. and Woods, P. (1998) Competition as a spur to improvement? Differential improvement in GCSE examination results, presented to International Congress for School Effectiveness and Improvement, Manchester

Lieberson, S. (1981) An asymmetrical approach to segregation, in Peach, C., Robinson, V. and Smith, S. (Eds.), Ethnic segregation in cities, London: Croom Helm

Marshall, G., Swift, A. and Roberts, S. (1997) Against the odds? Social class and social justice in industrial societies, Oxford: Clarendon Press

Robinson, P. and Oppenheim, C. (1998) Social Exclusion Indicators, London: Institute of Public Policy Research

Slater, J., Dean, C. and Brown, R. (1999) Learning gulf divides rich and poor pupils, Times Educational Supplement, 2/4/99:1

Social Research Update is published by:

Department of Sociology
University of Surrey
Guildford GU2 7XH
United Kingdom.

Telephone: +44 (0) 1 483 300800
Fax: +44 (0) 1 483 689551

Edited by Nigel Gilbert.

Autumn 1999 © University of Surrey

Permission is granted to reproduce this issue of Social Research Update provided that no charge is made other than for the cost of reproduction and this panel acknowledging copyright is included with all copies.