English universities have been warned against “inherently inflationary practices” such as using multiple algorithms to determine a module’s mark and discounting a student’s lowest scores after two institutions were found to have breached rules to stop degree inflation.

In a report on 6 November, the Office for Students (OfS) highlighted concerns over how universities in England were applying algorithms to calculate undergraduates’ final degree mark – with some institutions operating two or more algorithms and then awarding a student the highest class of degree out of all those determined.

Additionally, institutions that allowed academics to discount the lowest marks achieved by students over an assessment period also risked misrepresenting a student’s level of achievement, explains the study.

“Our view is that such steps are inherently inflationary, and that providers should move away from the use both of steps that discount students’ credit with the lowest marks and of steps that take the best result from a choice of multiple algorithms,” it states.

����Vlog

The regulator has begun investigating grading practices amid growing concerns over a steep rise in the number of first-class and upper second-class degrees that was felt unlikely to simply reflect improved teaching standards.

The number of students awarded first-class degrees at English universities was 15.8 per cent in 2010-11 but had “more than doubled” to 37.7 per cent in 2020-21, before falling back to 28.8 per cent in 2023-24, according to the OfS’s  on degree inflation, also published on 6 November.

����Vlog

That had prompted a series of investigations into individual universities where degree results had risen sharply immediately after the introduction of new academic regulation rules.

At St Mary’s University, Twickenham, the number of students receiving first-class and upper second-class degrees rose by 11.4 percentage points between 2015-16 and 2016-17, and by 4.1 percentage points in the following year, after two degree algorithms replaced a single classification algorithm, the OfS explains in 

Those increases came after the university set objectives in its strategy relating to students’ attainment of “good degrees”, targeting an increase in the number of such degrees by 13 per cent (to reach 73 per cent) by 2025, says the OfS report.

Modelling by the OfS revealed the new algorithm resulted in 79.5 per cent of students receiving a first-class or upper second-class degree in 2016-17, 4.7 percentage points higher than if the old algorithm system had been used on the same student cohort. Some 35.8 per cent of students received a first under the new algorithm system, up from 28 per cent if the old system had been used that year, says the OfS, which used modelling on a representative sample of students to compare results.

At the University of West London, algorithm changes in 2015-16 included the discounting of the lowest 20 credits, changed weighting of marks and more generous rules for students on the “borderline” of classifications. These changes meant 85.3 per cent of students received either a first-class or upper second-class degree compared to 81.7 per cent if the old regulations had been used, said the OfS , which based its reasoning on a representative sample of 856 students from 2015-16.

At both St Mary’s and University of West London, the continued use of these academic regulations for at least some students after May 2022 (when new OfS regulations took effect) meant both institutions had breached the OfS’ B4 conditions that are designed to ensure institutions have regulations that guarantee qualifications are “credible”, the regulator said.

However, the two institutions’ engagement with the OfS and their introduction of new academic regulations prior to 2022 to remedy concerns had led the regulator not to take further action, it said.

����Vlog

A third institution, Teesside University, where degree attainment also rose after the introduction of new academic regulations in 2014-15, was 

����Vlog

The rise in “good degree” results of just 1.9 per cent when old and new algorithm systems were tested against each other was “not statistically significant”, the OfS said.

It found the institution was, however, “at risk of breaching condition B4” over its continued use of “best grade” approach using multiple algorithms, noting the university had agree to undertake further checks to ensure its awards represented student achievement.

The intervention by the OfS on the issue of degree algorithms represents a significant step for the regulator, which acknowledged it had “not previously published our views relating to the use of algorithms in the awarding of degrees”.

Institutional practices have so far relied on “sector guidance [which] has consistently expressed concern about the use of multiple algorithms”, explained the OfS. Views on discounting of marks is “more mixed”, the regulator flagged its concerns on this practice, stating it “will be harder for a provider to demonstrate compliance with condition B4 when using one of both of these steps in its algorithm”.

Josh Fleming, head of strategy and delivery at the OfS, said the publications reflected a new era of engagement with the sector on practices likely to lead to degree inflation.

“We cannot say how widespread these practices are, but we know it goes beyond the three providers in these reports,” he said, noting the report sets out the need for institutions to test changes to the algorithms they use to determine degree classifications.

Jean Arnold, deputy director of quality at the OfS, said it was “essential that students, graduates, and employers are confident that higher education qualifications represent an accurate assessment of achievement that stands the test of time”.

“We welcome the progress made by institutions in their efforts to curb the risk of unexplained grade inflation. In recent years, the sector has reduced the proportion of students awarded first-class and upper second-class degrees, although levels remain above those awarded in 2010-11.”

����Vlog

jack.grove@timeshigheducation.com