Two years ago, the accountancy firm EY made an announcement that no doubt sent a shiver down many lecturers鈥 spines. After failing to find any published evidence that graduates with good degree results made for better employees, a trawl through its own data, the company revealed, similarly found 鈥渘o evidence to conclude that previous success in higher education correlated with future success in subsequent professional qualifications undertaken鈥.
Instead, what predicted success at EY, defined by recruits鈥 performance in annual appraisals and accountancy exams, is a 鈥渕ix between behavioural and cognitive attributes鈥, explains Dominic Franiel, head of student recruitment at the company. These attributes include many things that higher education is supposed to instil. Among them are logical thinking, ability to understand the root cause of a problem, rapid comprehension of new concepts, self-motivation, a confidence-inspiring and professional manner and a strong work ethic, Franiel explains.
Acting on these findings, the company, which recruits about 900 UK graduates alone each year, scrapped its requirement for applicants to its graduate scheme to have an upper second-class degree and 300 Ucas points (equivalent to three B grades at A level). They still need a degree, and the company 鈥渟till values academic achievement鈥, says Franiel. But rather than relying on university grades, EY now assesses applicants using its own bespoke tests. And this has had a noticeable effect on the kinds of people being successful 鈥 with a of graduates who went to state schools and who were the first in their family to go to university, says Franiel. Nearly one in five new recruits would have been excluded under the old system for having low grades, he adds.
EY鈥檚 move is part of a growing trend. Another UK-headquartered accountancy firm, , also scrapped its 2:1 requirement in 2013. In the same year, 鈥檚 internal research discovered that college grade-point averages and other test scores were 鈥渨orthless鈥 in predicting future success at the company, and scrapped its requirement for applicants to submit detailed results for everyone except brand-new graduates. And last year, the publisher dispensed with the need for a degree altogether, saying that there was no link between having one and workplace performance. According to Stephen Isherwood, chief executive of the UK鈥檚 Association of Graduate Recruiters, 鈥渢here鈥檚 a growing recognition that using academic grades as a kind of binary cut-off, as a cliff edge, is no longer [effective]鈥.
糖心Vlog
Few would argue that universities should simply be factories for the production of trainee accountants able to fit seamlessly into the corporate world. Higher education is clearly about much more than that. But there is increasing scrutiny of graduate outcomes by both governments and students (the UK鈥檚 teaching excellence framework and Longitudinal Education Outcomes project are good examples). If higher education is not seen to be adding any value to graduates鈥 skill sets, that has clear implications for university enrolment and funding levels.
So what value is higher education supposed to add? And how is this different from what school or vocational education offer? When challenged on this, the stock response from university leaders is 鈥渃ritical thinking鈥. Although the term is rarely defined in great detail, it is understood to involve an ability to think independently and to question assumptions in a structured, logical way. And, quite apart from the usefulness of such an ability in the professional sphere, it is also seen by many as a crucial element of informed citizenship. Insufficient levels of it among the electorate have been blamed by some for the success of last year鈥檚 campaigns for the UK to leave the European Union and for the US to elect Donald Trump as president 鈥 which, opponents say, made claim after claim that bore scant relation to logic or truth.
糖心Vlog
But there are two related questions around this. One is whether firms themselves really value the kind of critical thinking that academics prize. The bespoke tests that they are increasingly using to sift job applicants suggest that the kind of contemplative, expansive anatomisation of arguments that leads to high essay grades may not be what is required by successful operators in the modern, time-pressed workplace.
A further question is whether even the academic brand of critical thinking is being particularly well taught at university. According to Bryan Greetham, a philosopher and university researcher who has written several books on how students and professionals can improve their thinking, 鈥淲e tend to want to do the simple thing 鈥 which is to teach students what to think, not how to think.鈥 And it has long been an open secret in higher education that the sharpening effect, however defined, of a university education on students鈥 minds is far from well evidenced.
This was most famously explored in the 2011 book Academically Adrift: Limited Learning on College Campuses. The authors, American sociologists Richard Arum and Josipa Roksa, found that 45 per cent of US undergraduates failed to significantly improve their critical thinking, complex reasoning and writing skills during their first two years at university. Other US-based studies have raised similar concerns. One from 2009, 鈥溾, published in the Journal of Experimental Education, warned that college and high school students have 鈥渄ifficulty evaluating arguments on the basis of their quality鈥.
But definitive studies of the issue are still lacking. There is, for instance, no university equivalent of the Organisation for Economic Cooperation and Development鈥檚 Pisa tests for schools, which measure reading and numeracy skills at different ages across countries. When the OECD recently trialled a 鈥 of graduates, it was effectively scuppered after opposition in the UK, the US and Canada.
So, with the credibility of their credentials questioned, what can universities do to ensure that their students graduate with better cognitive skills? How can they live up to their claim that critical thinking is their unique selling point?
Some have begun explicitly teaching analytical skills. For instance, Anne Britt, a psychologist who has conducted studies into whether students can comprehend, evaluate and write arguments, runs a course at Northern Illinois University called simply 鈥渢hinking鈥. After just half an hour of instruction, she says, most students do show significant improvement 鈥 although there is also a significant minority who make little progress even after a term.
To get the hang of certain ideas, she says, requires plenty of contact time between student and teacher. 鈥淚 have many students who come in and think they鈥檙e excellent at argumentation. In fact, they鈥檙e not. Early on, they need feedback because they conflate argumentation with giving an opinion,鈥 she says.
But if universities don鈥檛 have the resources to offer intensive classes, could they weave the teaching of critical thinking skills into regular teaching? Britt thinks that academics can easily make time for quick 鈥渃heck-ins鈥 during their lectures to ensure that their students understand what they鈥檝e been told. 鈥淚t doesn鈥檛 have to take long. It could take 10 minutes at most鈥therwise students are not actually thinking during the lecture,鈥 she says. A quick intervention can make students realise: 鈥淥h! This is how I鈥檓 supposed to be reading my book. These are the kinds of questions I鈥檓 supposed to be asking.鈥
糖心Vlog
High school experience, of course, varies enormously by country. In France, studying philosophy 鈥 arguably the closest that traditional disciplines get to explicit critical thinking courses 鈥 . In England, meanwhile, the critical thinking A level has . And Robert de Vries, a lecturer in quantitative sociology at the University of Kent, became 鈥渃onvinced pretty quickly鈥 that many UK students need 鈥渆xplicit, remedial instruction in these abstract skills. I get the sense that students are used to being marked for content 鈥 鈥楬ave you covered this topic? Have you mentioned this fact from the textbook?鈥 鈥 rather than for the quality of their reasoning or argumentation.鈥
However, according to de Vries, university courses 鈥渙ften can鈥檛 devote the time needed to explicitly teach the abstract tools of critical thinking: how to construct a good argument, how to spot weak evidence for a claim. They have a lot of substantive content to cover, and, to an extent, they have to assume that students will have already picked up a lot of this stuff by the time they get to university.鈥
This state of affairs, he says, explains why he too teaches a specific critical thinking course, which is compulsory for Kent鈥檚 sociology, social policy and social research students, and can be taken as an optional module by other students.
Another problem with assuming that students can pick up critical thinking skills during their normal studies, Britt adds, is that each subject leaves them with very different ideas about how to argue. In contrast to the scientific approach, 鈥渋n history, I might never have a graph of data鈥, she says. Hence, when graduates start work, a historian and a scientist may begin with very different concepts of what constitutes reliable evidence, she says. So the idea that all university graduates have a generic ability to think critically may be somewhat misleading.
Moreover, widespread doubts exist that critical thinking is the be-all and end-all of employability. As previously mentioned, employers additionally value certain attitudinal traits, which their aptitude tests also seek to test. And, according to Greetham, critical thinking is not enough to enable graduates to do what their employers prize above all: the ability to come up with new ideas and concepts, and to create solutions to problems.
糖心Vlog
For him, critical thinking is useful, but it 鈥渨orks on the assumption that facts, right answers and certainties are out there just waiting to be discovered鈥 by logical thinkers. 鈥淚t assumes that a teacher鈥檚 role is to find these facts and transmit them, while teaching students the skills to chip away all those things that obscure them, [such as] the inconsistencies in our reasoning, irrelevant arguments and unsupported assumptions.鈥
But, this, Greetham says, is a flawed picture of reality. Rather than creative thinking, he says, students need to be taught what he calls 鈥渟mart thinking鈥. This requires lecturers to allow far more discussion in class, and to guide students in how to analyse and synthesise concepts. 鈥淵ou need to lose control in the classroom in a way 鈥 because you鈥檝e got to say: 鈥楲et鈥檚 have your ideas,鈥欌 he argues. Instead of requiring students to present their ideas to the class, followed by a discussion, lecturers need to get students to analyse the meaning of concepts, 鈥渞ecording their ideas on a whiteboard as they shape and reshape them鈥, and guide them in 鈥渞einterpreting and adapting their solutions鈥. This kind of teaching, he maintains, genuinely develops creative and conceptual skills.
But, Greetham warns, such change is unlikely to be possible while academics continue to be recruited on their research record, as opposed to their teaching ability.
As well as calls for critical thinkers and smart thinkers, there are also from politicians for more 鈥渆ntrepreneurial鈥 university graduates 鈥 who, instead of joining graduate recruitment programmes at large employers, might start their own businesses. Exactly what this means is hard to pin down, but it generally involves an emphasis on less theory and more practical experience.
Indeed, there is a general move across the world to align higher education more closely with vocational training. Nowhere is this trend more apparent than in Germany. 鈥淵ears ago, the two education sectors were following different, opposite, logics 鈥 scientific versus practical orientation 鈥 and were addressing strictly separated job markets,鈥 says Ulrich Mueller, head of policy studies at the Centre for 糖心Vlog, a German thinktank. 鈥淭he situation now is different: there is a big overlap between academic and practical education.鈥 There are now very job-focused academic courses, he says, as well as vocational programmes with lots of academic content. The overlap is such that graduates of these two streams now compete for the same jobs in some areas, such as medicine and computer science. 鈥淚 am sure that in 15 or 20 years there will be an [integrated] system of post-secondary education,鈥 Mueller concludes.
In such a world, of course, there would be no need for higher education to define itself in a particular way, in contrast to other forms of education. But in the meantime, pinning down the unique skills of university graduates remains moot 鈥 especially when accessing higher education can be so much more expensive than vocational alternatives.
It is possible to overstate the level of concern about the issue. The US National Association of Colleges and Employers, for instance, says that it hasn鈥檛 seen a shift away from degrees towards employer-designed tests, suggesting that US firms largely still see college GPA as a reliable indicator of the kinds of skills they are looking for. And, in the US, getting a college degree is arguably in terms of being competitive in the job market.
Yet, paradoxically, the bigger the graduate cohort becomes, the more employers are likely to question exactly what, if anything, having a degree really indicates. And that, in turn, could require universities to ask hard questions about how much 鈥 or how little 鈥 their courses really change how their students think.聽
Think fast: what do graduate recruitment tests actually involve?
Having taken several of the bespoke tests that firms are now using to put their graduate applicants through their paces, it鈥檚 probably fair to say that I聽shouldn鈥檛 give up the day job.
But I聽suspect that I聽am not the only degree-holder who would struggle: the tests assess skills rather different from those required to pass university exams. Some questions involve numbers, shapes or text, but all are broadly designed to gauge mental agility. In abstract reasoning tests, for example, a typical question features a sequence of shapes, which you are asked to continue. Circle, square, circle, square: what next? That鈥檚 easy 鈥 but the patterns quickly get a lot more complex than that.
Another spatial reasoning test designed for recruiters by specialist firm Saville Assessments involves rotating 3D shapes in your head to see which is the odd one out. Meanwhile, a verbal reasoning section requires you to speed-read a passage about eating habits 鈥 putting any emotional or critical reaction out of your mind 鈥 and then click on the best summary of it, or choose the best synonym to replace a word. Worryingly for a journalist, I聽completely flunk error-checking, which involves scanning a spreadsheet at a furious pace.
It is the rapid-fire nature of the tests that most distinguishes them from university work. You rarely have more than 30 seconds to answer a聽question; in some cases, you have little more than
10 seconds. It鈥檚 a different form of mental activity from, say, crafting a聽dissertation over many months.
But perhaps the skills that these tests assess 鈥 can you quickly skim a paragraph or a screen of numbers, and then fire off an acceptable answer? 鈥 are more useful
in the modern, time-pressed office than the sustained thought that higher education is supposed to -inculcate.
That said, it is important to note that the tests aren鈥檛 used in isolation. Situational judgement tests, where applicants are presented with a real-world dilemma, are also increasingly in vogue. According to Dominic Franiel, EY鈥檚 head of student recruitment, a candidate might be asked, for instance, how they would handle a situation in which a more senior colleague was suddenly called away and they were left having to meet clients. Given a list of five choices, 鈥渢here鈥檚 probably an ideal answer鈥, but it鈥檚 鈥渜uite nuanced鈥.
Moreover, applicants who get past the first online round do then have to weigh evidence and formulate their own opinions at a more leisurely pace at assessment centres, he adds.
糖心Vlog
It is also worth noting that the testers make no pretence to be assessing raw ability. They readily admit that scores can be enhanced by effective prepping 鈥 which I聽did not do. So perhaps there still hope for more ponderous graduates like me after all.
David Matthews
POSTSCRIPT:
Print headline:聽Fuzzy logic
Register to continue
Why register?
- Registration is free and only takes a moment
- Once registered, you can read 3 articles a month
- Sign up for our newsletter
Subscribe
Or subscribe for unlimited access to:
- Unlimited access to news, views, insights & reviews
- Digital editions
- Digital access to 罢贬贰鈥檚 university and college rankings analysis
Already registered or a current subscriber?
