In December, the Higher Education Funding Council for England (HEFCE) released a document[1] aimed at students to explain what the Teaching Excellence Framework (TEF) is, writes PHD student Arte Artemiou. In it we see statements claiming that the TEF will “give students clear information about what teaching quality is” and to “encourage a stronger focus on the quality of teaching in higher education”.
These statements seem to be misleading and considering this is a document for students, that’s worrying. Dr. Andrew Gunn, a researcher in higher education policy, has said[2] that the TEF “is assessing everything that goes on before admission – from outreach, choosing candidates and interviews – to the actual teaching environment and then everything that students do after graduation.”
So not simply teaching excellence but the “entire teaching function of a university”.
Are these aims possible using the metrics that the TEF has set out? These metrics are: the National Student Survey (NSS), retention rates, graduate employability, and a 15-page narrative statement on the institutional context of the university. The 15-page narrative statement will be used as the deciding factor of whether a university can be upgraded or not after a provisional rating is given according to the other three metrics. All of this is being measured so that a university can charge future students more money depending on whether they achieve bronze, silver, or gold (although this is thought to be in line with inflation).
The NSS
Research[3] into the 2014 results of the NSS found that “students are happiest on degree programmes where high proportions of staff have the following characteristics: white, full professors, holding doctorates, and on fixed-term contracts” at universities that either rated research highly or had a high proportion of professionally qualified faculty found that which has very little to do with teaching excellence. As a result, the researchers questioned whether students, when completing the NSS, failed to see the impact that research may have on their studies and the potential that going to a more ‘elite’ university may instil expectations which are not found at non-elite universities, thus increasing the dissatisfaction scores of the NSS.
These findings and musings call into question the validity of the NSS as a metric for teaching excellence, let alone for the teaching function of a university. Even the Chair of the TEF’s assessment panel has admitted[4] to the inadequacy of the NSS stating that, “I do not think student satisfaction is an accurate proxy for teaching quality”, yet in the same breath claims that this doesn’t mean we mustn’t use the NSS. He argues that we must discover how the data it provides could be used “effectively”. Surely claiming that the NSS can’t be used as a useful metric but also claiming we should still give the results validity in some way is contradictory?
Retention rates
Research[5] into retention rates at universities found that those from disadvantaged backgrounds were more likely to leave before completion of their studies and those who went to universities with lower UCAS requirements were more likely to withdraw from their courses. According to the research, there were universities effectively challenging these trends, with those achieving low UCAS entry points and coming from disadvantaged backgrounds remaining at university at various interventions but low UCAS points and socioeconomic status isn’t the fault of universities so why is it a metric of teaching excellence?
Other research[6] indicates that the problem is not with universities but with secondary schools. If students were better prepared for the pressures and expectations of university they would either:
- a) not attend because it was not something they wanted to do, or
- b) already be developing the skills required to engage with undergraduate studies.
The very nature of ‘paying’ for an education (which is, in fact, borrowing money from taxpayers) has had a negative impact[7] on poorer students’ choice of university which could be also affecting retention rates as it isn’t their first choice.
Also, research[8] has shown that some courses are not worth the cost when compared to graduate earnings. Inevitably, students may not feel their ‘education’ is worth the amount of money they are borrowing to pay for and stop attending these courses. This is not solely the fault of universities but of the policymakers (like those who created the TEF) who are pushing an ideological vision of what education must be in the U.K.
Graduate employability
Research[9] into whether there are too many graduates in the UK found that graduates are filling jobs that were originally filled by school leavers; whether their degree is relevant to the position or not. This inevitably creates a false hierarchy of ‘educated’ vs the uneducated amongst employers, considering both will need to be trained in a graduate-irrelevant job. I am a firm believer that education – knowledge – must not be commodified or turned into a means to an end (i.e. a job). Knowledge and education must be made available to all. The argument that there are too many graduates for graduate related jobs is true, hence the mismatch of skills required and degree obtained – but this doesn’t mean the answer is to limit the people going to university or to funnel more people into university. It means we need to address the role of the university. It must be a centre of knowledge and learning, not an attempt to mark a step on career progression. There is a pressure on the education sector that need not be there: a notion that university will better the lives of those who attend and that we must fit as many students as possible into university for them to ‘better themselves’ (i.e. graduate relevant jobs). It isn’t working[10].
15-page contextual narrative
This doesn’t need much attention. One simple point: giving universities the ability to colour their function in a way that will inevitably paint them in a positive light, even if that isn’t the reality, is like giving someone a test and then asking them why they believe they should pass.
What’s your point?
Most of the issues with satisfaction scores, retention rates, and graduate employability come down to the idea that education is a commodity to be bought and sold to further careers rather than having a right to exist on its own merit.
These metrics of ‘teaching excellence’ are heavily influenced by factors that the metrics don’t account for. If a university doesn’t meet a student’s lived experience of education, for example, then they may be dissatisfied and/or drop-out.
Therefore, the metrics for the TEF make no sense in regards to teaching excellence no matter how you spin it (even by changing the definition of ‘teaching excellence’). We shouldn’t be making the best of the metrics, as the Chair of the TEF assessment panel said regarding the NSS, but we should be trying to figure out what higher education needs to be and how to measure teaching excellence.
In conclusion
The NSS is subjective and insufficient, drop-out rates are affected by background and scores in secondary school, and employability can be factored down to whether an employer believes a degree is a good thing (regardless of job relevance).
And this information is being used to charge more money to ‘learn’?
Think about that.
1] http://www.hefce.ac.uk/lt/tef/guidance/
[2] https://www.timeshighereducation.com/news/teaching-excellence-framework-is-about-much-more-than-teaching
[3] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2712412
[4] https://www.timeshighereducation.com/news/tef-limit-use-flawed-nss-scores
[5] http://www.smf.co.uk/publications/staying-the-course/
[6] https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/317276/RR353__The_link_between_secondary_school_characteristics_and_university_participation_and_outcomes_FINAL.pdf
[7] http://www.educationopportunities.co.uk/wp-content/uploads/DoesCostMatter_ANEONReport.pdf
[8] https://www.ft.com/content/f326e3f4-00b6-11e6-99cb-83242733f755
[9] https://www.theguardian.com/money/2016/oct/11/huge-increase-in-number-of-graduates-bad-for-uk-economy
[10] https://www.theguardian.com/commentisfree/2016/apr/26/university-graduating-non-graduates-career-skills
