Campaigning for next May’s elections to the Scottish Parliament will doubtless draw widely on published educational statistics. But these are deeply flawed. A report by Enlighten a year ago explained this in detail, and proposed ways in which the situation could be improved. No political party has yet said what they would do about it.

Here are seven examples of misleading statistics. When you hear politicians making claims based on data of any of these kinds, challenge them to explain why they trust numbers that are so suspect.

Attainment in secondary schools

The claim that is heard every summer – and celebrated by government and opposition alike – is that attainment is stable or rising, and that the difficulties caused by Covid have been put behind us. On the face of it, the data seem to bear this out. One favourite criterion is the proportion of school leavers who had five or more passes at level 5 – that is, at National 5 or equivalent. In 2024 this was 55%, essentially the same as in 2012. No progress, but no apparent harm from those things that politicians cite as deeply harmful – Covid, economic difficulties, Brexit. For the next level of examination above that, there even seems to have been an improvement. In 2012, the proportion of leavers who had passed three or more Highers (or equivalent) was 40%. In 2024, this rose to 42%.

There are of course two ways in which a country’s examination attainment can appear to be strong. One is real learning. The other is a weakening of the standards of assessment. Objective criteria that are defined independently of the country’s own measures can distinguish between these two. When we do that here, the Scottish picture is not good. The only available international criteria are from the so-called PISA studies, run by the Organisation for Economic Cooperation and Development. In all three measures in PISA, the attainment of Scottish 15-year-olds has emphatically fallen since 2012. The most useful yardstick to say how bad the fall has been is equivalent years of schooling. In reading, between 2012 and 2022, the fall was about three quarters of a year. In both mathematics and science, it was a year and a half.

The age for the PISA assessments – around 15 – is much the same as for the Nationals, and only a year lower than for Highers. If the attainment of 15-year-olds has been falling in the way that PISA suggests, then the only reliable conclusion is that the assessment standards of the Nationals and Highers have weakened.

So the political claims of rising attainment, and of no lasting Covid harm to attainment, are statistical fictions.

Achievement of Curriculum for Excellence Levels

The same is even more true of Scottish government reports of attainment in early secondary and primary. These are published annually in December as ‘Achievement of Curriculum for Excellence Levels’. They appear to show a system that is doing extremely well, even better than in the statistics on Highers and Nationals. The earliest school session for which the information was published was 2017. For pupils in the third year of secondary school in that year, 90% had achieved the stipulated level in reading, and 88% in numeracy. In 2024, the corresponding figures were both 90%.

Pupils in the third year of secondary school are aged 14-15. Again, they are broadly the same age as for the PISA studies. So these attainment statistics, too, fail to capture the serious decline of attainment that has been picked up by PISA. They also fail to record any impact of Covid.

The reason for the discrepancy between the reported achievement and the PISA results is not difficult to find. What is euphemistically called pupils’ achievement of the curriculum levels is in fact teachers’ impressions of whether their own pupils have achieved the levels. These impressions are not based on objective data of the PISA kind. They are not even based on the somewhat more reliable data of the kind represented by Nationals and Highers. They are simply hunches. None of us, as teachers, are very good at assessing our own students on the basis of informal judgement. That’s why education has developed objective testing over the past century and a half, independent of teachers and of politicians. The Scottish government used to check regularly on the discrepancy between teacher judgements and real attainment, through the Scottish Survey of Achievement. It has not done so since they abolished that survey in 2008.

So the political claim that almost all pupils reach satisfactory levels of attainment is our second statistical fiction.

Social inequality

The present Scottish government says that closing what it calls the ‘poverty related attainment gap’ is its educational priority. No other political party dissents from that aim.

The published statistics seem to show that progress is being made. The regular series on this topic reports attainment in terms of the social characteristics of the neighbourhood in which pupils live. These characteristics are summed up in the Scottish Index of Multiple Deprivation, which for publication is grouped into five categories, ranging from the most deprived fifth to the least deprived fifth. Inequality of attainment at a particular level is then measured as the difference between the proportions in these two groupings who gained that level. For passing at least one National 5 or equivalent, the inequality was 27.4% in 2012, but 18.1% in 2024. For Higher or equivalent, it fell from 43.8%to 31.8%. Over a decade – especially one that included the enormous disruption caused by Covid – the fall of inequality might seem quite heartening.

But this whole approach mis-uses the Index of Multiple Deprivation. It is a genuinely valuable way of understanding the characteristics of whole neighbourhood. But it is misleading as a measure of the social circumstances of individual pupils and their families. That’s because the ‘neighbourhoods’ on which the Index is based are internally heterogeneous. There are many non-deprived people living in the most deprived neighbourhoods (and there is much hidden poverty in the most advantaged places).

We can estimate the extent of the inadequacy by using the high-quality annual Scottish Household Survey. The most recent survey from which the dataset is available is 2022. This survey reveals large pockets of social advantage even in the 20% most deprived neighbourhoods. For example, fully 18% of families with children in these deprived neighbourhoods contained at least one university graduate. One quarter of families with children in these neighbourhoods had annual household income of more than £40k. The corresponding proportions in 2012 were much lower – 10% and 5%.

The problem is that the improved attainment by pupils in the most deprived neighbourhoods might well be due entirely to the growing minority from these non-deprived families. Children in families with a graduate parent tend, on average, to have much higher attainment than other children. So the apparent fall of inequality might not be due to any relative progress by children in actually deprived families.

The PISA studies again allow us to assess what has truly happened to individual inequality. PISA contains an index of the social circumstances of individual families, not neighbourhoods. Inequality can then be measured by the gap in PISA attainment between the most and least advantaged quarters of that index. In 2012, in reading, the gap was equivalent to 3.9 years of schooling; in 2022, it had widened to 4.7 years. In mathematics, these gaps were 4.3 and 5.2. In science, they were 4.1 and 4.8. The consistently growing gap, when measured properly in this way, adds strongly to the sense that the progress painted by the Scottish government’s statistics is an illusion.

An ostensible narrowing of inequalities because graduates increasingly live in socially deprived neighbourhoods may tell us something about the dire state of the graduate labour market. But it does not, on its own, tell us about the quality of the schools. So this political claim of falling social inequalities of attainment is our third statistical fiction.

Attendance

Statistical reports on pupils’ absence from school seem to show no more than a quite minor problem. They show, for example, that in primary school, on any given day in 2023-4, around 7.5% of pupils were absent, and only 2.6% were absent without permission (‘unauthorised absence’). The picture was somewhat less rosy for secondaries, at 12.3% and 5.2%, but even these could not be described as a crisis.

But the real crisis is persisting absence. The one in twenty secondary pupils who are unaccountably absent on one day might or might not be the same as those who are absent the next day, or the next week. Enlighten’s Commission on School Reform found in 2023 (but only through Freedom of Information Requests) that there was an alarmingly high proportion of pupils who were absent for lengthy periods of time. Around a quarter of pupils at primary school, and four out of ten at secondary, were absent on at least 10% of occasions. Following that report, the Scottish government started publishing annual information on this, including some historical data that confirmed Enlighten’s research. This is indeed a crisis. No wonder attainment (when measured properly) is falling.

So the political claim of generally good attendance is our fourth statistical fiction.

Global citizenship

Faced with the indubitable evidence of declining attainment in PISA, Scottish politicians like to cite the apparently better Scottish showing in a test of ‘global citizenship’ that was part of the PISA study in 2018. Compared to other countries, Scottish 15-year-olds did perform relatively well in these tests, but only 27 countries took part (whereas 78 took the tests of attainment). These did not include any of the high-attaining European countries – such as Estonia – nor any of the other parts of the UK. A rather larger group of countries did use the accompanying survey which asked for students’ attitudes to global issues, and for whether these were taught in schools. This included 20 EU countries. Scotland had high proportions with liberal attitudes. For example, large majorities of Scottish 15-year-olds had positive attitudes to immigrants, for example supporting equal opportunities for immigrants, their right to vote, their right to a distinctive culture, and their general rights. On each of these questions the proportion in support of the liberal position was around 88% in Scotland but around 13 points lower across the participating EU countries.

But the Scottish pupils were less likely than elsewhere in the EU to report that they were learning about global challenges from their school. For example, for learning about different cultures in school, the proportions were 70% in Scotland but 76% across the other EU countries. For learning about solving conflicts, they were 52% and 61%. For learning about the interconnectedness of economies, they were 40% and 56%.

So the claim that Scottish adolescents are liberal and globally minded is quite accurate. But the political claim that this is due to the schools is our fifth statistical fiction.

Positive destinations

In Scottish-government parlance, a positive destination for a school leaver is having something worthwhile to do. According to the annual reports on these, the proportion who enter them is extremely high – over 95%. But the definition of positive is amorphous, being ‘higher education, further education, training, employment, voluntary work, [or] Personal Skills Development’. Employment might in theory be a stable, full-time job with good prospects. But for a school leaver who is not progressing to training or education, it is much more likely to be a zero-hours arrangement with no actual contract and no offer of training or any other kind of career advice. If we narrow down to the destinations which have some form of quality control – higher and further education, and training – the numbers are far lower. Recently, around 30% have not been in these three categories.

Again, as with attainment, the problem is compounded when we are trying to understand social inequalities. Using the Index of Multiple Deprivation, the proportion of leavers in 2024 with this narrower definition of positive destination was 70% from the most deprived fifth of neighbourhoods and 79% from the least deprived, a gap of 9% that was a quarter less than the 12% of 2012. That drop was mainly due to a rise in the most deprived fifth from 64%. But as with the attainment measures, this change of inequality since 2012 could be entirely explained by the growing proportion of families in deprived neighbourhoods who have a university graduate in the household. Graduate parents not only help with a child’s learning. They can provide advice on career paths, on how to apply to university, on what kinds of voluntary work can eventually lead to a good job. Therefore, once again, the published statistics do not tell us whether the apparent fall of inequality is an illusion.

So the political attributing of positive destinations to anything which schools are doing is our sixth statistical fiction.

Relationship of per-pupil expenditure and outcomes

The final topic is about inputs as distinct from outputs. The Scottish government spends more per pupil than England. That is affordable only because of the flow of tax money to Scotland from the rich parts of England.

A study earlier this year by the Institute for Fiscal Studies quantified the gap:

‘in 2009-10, spending per pupil in Scotland was around £8,300 in today’s prices, which is about 4% higher than the £8,000 seen in England at the time. By 2024–25, spending per pupil is expected to be £10,100 in Scotland, approximately 20% higher than in England (£8,400 per pupil).’

The IFS authors point out that much of this relative increase in Scottish spending was taken up with salaries for teachers. The overall growth of 27% between 2015-16 and 2023-24 included three 7% increases in teachers’ pay (2019, 2022 and 2023).

However, spending tells us little about the quality of a school system. Even if we were to take the Scottish government’s version of the statistics quoted above, we would still have to conclude that the increase of expenditure had led to only a rather modest increase in attainment or ‘positive destinations’. When we use objective data, and when we also compare that with England, the inefficiency of Scottish public spending in education is stark. In PISA, comparing 2012 with 2022 (thus including the Covid disruption), whereas Scottish reading scores fell by the equivalent of three quarters of a year, the English fall was only one fifth of a year. In mathematics, the year and half fall in Scotland was about nine times greater than the fall in England. In science, the fall in Scotland (a year and a half) was more than twice the fall in England. The well-funded Scottish schools, with well-paid teachers, did not provide any greater resilience during the pandemic than the less generously funded schools in England.

So the political boast of generous public spending on education in Scotland is beside the point. Claims about the value of spending lots of money is thus our seventh statistical fiction.

In short, despite misleading data, attainment is probably not satisfactory, policy is probably not causing inequality to fall, destinations for about a third of school leavers are not, in any genuine sense, ‘positive’, and lavish expenditure is, in large part, a waste of money. So when confronted over the next six months by canvassers or candidates or manifestoes – or even by ostensibly neutral publications from the civil service – be sceptical. And perhaps recommend Enlighten Scotland’s plan for better educational statistics.

Lindsay Paterson is Professor Emeritus of Education Policy at the University of Edinburgh