Autumn Statement 2015

As in previous years, the Chancellor’s Autumn Statement has provided a clear steer for the future of HE. For those who don’t see HE as being driven by the Treasury view, then look no further to the announcement in 2013 on the scrapping of Student Number Controls.

This year the headlines for HE are:

International students: numbers to grow;dependants of postgraduates on courses lasting more than a year will be welcome to come and work.

Widening participation: work with the Director of Fair Access to ensure universities take more responsibility for widening access, including collaborating on outreach to reduce inequality in admissions

Postgraduate support: lift the age cap on new loans to postgraduates from 2016-17 so they are available to all those under 60.

Part time support: introduce new part-time maintenance loans from 2018-19 to support the cost of living while studying.

ELQs: For all STEM subjects, tuition loans will be extended to students wishing to do a second degree from 2017-18

Widening the range of providers: £20 million competition to set up a new Institute of Coding; create a new university in Hereford focused on engineering in 2016; help to fund the £100 million development of a new campus in Battersea for the Royal College of Art.

Research: protecting today’s £4.7 billion science resource funding in real terms; long term science capital commitment of £6.9 billion between 2015-2021 to support the UK’s world-class research base; introduce a new body – Research UK – which will work across the seven Research Councils, review of the Research Excellence Framework

For detailed analysis of the figures, and in particular the implications for the changes to student loans , then read Andrew McGettigan over on Wonkhe.

So – area of concerns: two leap out straight away, firstly, there is the change to support for students in nursing and midwifery from the 2017 intake, who will now access loans in the same way as other student. This may have an impact on attractiveness of such courses, although in other areas the change to funding did not, overall, lead to a decline in applicants.

The second area of concern is around the student opportunity fund. The focus on social mobility is strongly articulated in the recent Green Paper, and the opportunity for funding for part time students and those wishing to study WLQs can be seen as part of the same narrative. A worry for a while has been that the Student Opportunity Fund within the  BIS HE budget was an easy target   The fund in itself is safe for now but:

The government will work with the Director of Fair Access to ensure universities take more responsibility for widening access and social mobility, and ask the Higher Education Funding Council for England to retarget and reduce the student opportunity fund, focusing funding on institutions with the most effective outcomes

And again, this could be seen as part of the continuum with the Green Paper, where some of the proposal for the higher levels of Teaching Excellence Framework will reward those institutions who perform well on this agenda.

Those universities that receive significant amounts of SOF will be looking hard at their income and the possible related outputs – here I’ve drawn together the 15-16 funding allocations, with data from league tables on entry tariff, good degrees, and degree completion. More detailed analysis will be needed to consider how to measure part time students, but a few hours playing with Heidi data might reveal how individual institutions are performing

wp allocations

As I’ve written before when looking at TEF, this is another example of a great opportunity for institutions to get clever about how they can use data to measure their performance, not just for the sake of creating a set of metrics, but as a way of measuring the efficacy of interventions, and using this to provide better evidence-based decision making.

There are some opportunities in this autumn statement for us as well – for example the more welcoming statement about  international students, although this still conflicts with the broader negative views around immigration, and for us would rely on assuming that students in our target market have dependents.

Availability of loans for postgraduate study is to be welcomed, increasingly higher level qualification will be needed for certain routes into employment, or as ongoing staff development.

The changes to availability of loans for part time study should also be welcomed, as indeed should the opportunity of funding for ELQs in STEM subjects.

Overall – it could have been a lot worse for HE – the BIS budget wasn’t cut as much as predicted, but with the Green Paper currently out for consultation, we probably have enough on our plates to be going on with!

 

 

Presentation to Academic Group Leaders

We regularly hold a forum at Staffordshire University for our Academic Group Leaders – these are the senior academic staff who are responsible for line managing and leading groups of academic colleagues.

This week I led the forum, with a presentation on league tables and on some of the implications of the recent Green Paper “Fulfilling our Potential. Teaching Excellence, Social Mobility and Student Choice“.

The slides can be viewed here at Slideshare

 

I can’t get no satisfaction

Early in January the next round of National Student Survey begins across the UK HE sector. This year for many will be seen as  a dry run for what is to come in later years – the widely discussed  Green Paper which refers to using metrics to help gauge teaching excellence. Once we get past the first year of everyone being equally excellent, then  NSS and DLHE are widely anticipated to be key measures to be used, as well as possible measures of learning gain, since the paper does hint at a lack of satisfaction with current degree classification processes.

So before we check on progress on last year’s action plans, and start to think about how we introduce this year’s survey to our students, a couple of publications from the last week are worth bearing in mind.

Firstly a research paper from QAA, written in part by Jo Williams of University of Kent, to which Staffordshire University made  a contribution. In this, “The Role of Student Satisfaction Data in Quality Assurance and Enhancement: How Providers Use Data to Improve the Student Experience” Dr Williams looks at how different types of institutions approach NSS, and shows that across all parts of the sector, institutions and senior staff questions NSS:

In particular, the issue of question 22 of the NSS, asking students about their overall satisfaction has created endless debates in academia, if not confusion, with professionals arguing that it is methodologically and professionally wrong to base league tables on a single question which is not in itself clear. Various senior academics we spoke to concurred with this theme.

The research did show that universities in all parts of the sector listen to what students say, and that they do make changes based on what the survey reveals, for instance:

The programmes change every year so sometimes it’s because the subject changes but very often it’s because students have expressed discontent with something. Therefore, you change the personnel teaching it. You change the way you teach it. You change the content, you change the assessment, you change the feedback, you change something about it. Or sometimes you just drop it.
(University B)

Changes in practice were noted across institutions:

Our data revealed that institutions have employed various changes in response to issues students raise in the satisfaction surveys. Among other practical changes, universities have:
– recruited academic advisors and officers to take charge of the NSS
– mapped internal surveys to mirror the NSS
– renewed their focus on learning and teaching revisited and improved timetabling systems
– raised structures including building sites, teaching rooms and sports complexes
– revisited their feedback and assessment mechanisms
– organised briefings with students to enlighten them about feedback and assessment, the NSS and its benefits
– replaced subjects, at times personnel, whose NSS scores keep falling
– introduced or empowered various forums and platforms to meet several times in a year to discuss the NSS, among others. Such forums found at nearly all institutions include: NSS forums, student experience action plans, education boards, NSS improvement forums and learning and teaching advisory groups

Across the institutions in the research, other similarities were see in how data was used: for instance comparing scores across schools, holding low scoring schools to account, and comparing with other institutions.

In terns of league tables, depending on where you appear in a league table appears to influence the behaviour of the organisation.

In particular, institutions placed in the top 25% of the league tables appear to have a relaxed view of the NSS. They appear to put particular emphasis on improving the student experience and argue that this automatically triggers a higher satisfaction rate than being ‘obsessed with the NSS’ and improving league table position:

Whereas at the other end of the scale:

In contrast, institutions in the lower 25% of the student satisfaction league tables appear to place particular focus on improving their student satisfaction and subsequently their standings in the league tables.

The main conclusion of the work then is that:

In particular, institutions in the top 25% of league tables
(Universities A and B) appear to prioritise improving the student experience and let the NSS take care of itself, while those in the bottom 25% (Universities C and D) prioritise their NSS league table position and subsequently employ various tactics to promote the surveys.
Despite institutions adopting different approaches to the surveys based on league table positions, institutions generally listen to students’ demands raised in surveys and have responded by instigating various changes including recruiting academic advisers and officers to take charge of the NSS; mapping internal surveys to mirror the NSS; raising structures including building sites and revising their feedback and assessment processes.

What the paper doesn’t consider is the relative ranking of NSS scores by institutions – it is perfectly possible to score well on certain NSS scores, and appear to out perform other institutions on such a single measure, but this may not change institutional behavours which may be set to focus on the NSS position, rather than overall experience.

In other work out recently, from Stephen Gibbons, Eric Neumayer and Richard Perkins writing in the Economics of Education Review “Student satisfaction,league tables and university applications: Evidence from Britain” (S. Gibbons et al./Economics of Education Review 48 (2015)148–164), the authors make the following points:

  • NSS scores have an impact on recruitment applications, but not huge
  • students do not appear to respond directly to quality cues from satisfaction scores
  • students may already have a well developed knowledge about product quality based on perceptions of reputation and prestige
  • student satisfaction and league table position do not have a short term effect on market demand
  • the degree to which quality indicators affect demand is strongly linked to the amount of market competition for a given subject

Finally, in “Applying Models to National Surveys of Undergraduate Science Students: What Affects Ratings of Satisfaction?”  (Educ. Sci. 2013, 3(2), 193-207) by Langan Dunleavey and Fielding of Manchester Metropolitan University, the authors look at what influences the results seen for question 22 – overall satisfaction. We are all familiar with reading through a set of results, with great scores for most of the questions, and sections, but a lower score for this final crucial question, which is the one used in all league tables.

The authors note the year on year consistency of results for individual subjects, noting how comparisons should be made:

Subjects were highly consistent year on year in terms of their relative performance in the satisfaction survey. This has implications for institutional decision-making particularly if subjects are wrongly compared against institutional averages, when comparisons should be made within subject areas (e.g., comparing with national subject averages, although this may be subject to error if courses contain different compositions of learners, for example in terms of ethnicity)

This is consistent with HEFCE advice, and why as an institution we provide sector average scores at JACS3 subject level for comparison.

Interestingly, questions about feedback were the weakest predictors of “Overall Satisfaction” whereas:

The best predictor of student satisfaction nationally for the three years analysed was “The course was well designed and running smoothly” followed by ratings of “Teaching”, “Organisation” and “Support”. This may vary slightly between subjects/institutions, so it is proposed that this type of quantitative approach to contextualising survey metrics can be used to guide institutions in resource allocation to tackle student experience challenges.

So our conclusions on how we approach the next NSS, and perhaps more  importantly NSS2017  could be:

  • carry on listening to students, responding and being seen to respond to surveys
  • make sure we focus on all the measures that make up a league table
  • make sure that courses are well organised and running smoothly
  • don’t expect league table moves to immediately be reflected in increased applications
  • and remember – the student experience is what really matters, not the survey itself.

Wonk’s Delight

And so the waiting is over. Friday finally saw the long awaited release of one the  most anticipated artifacts of 2016. Twitter was inundated with commentary, and major newspapers rushed to add reports to their online editions as the new John Lewis Christmas advert became available.

In other news, the government published its green paper – “Fulfilling out Potential – Teaching Excellence, Social Mobility and Student Choice”.

Much of the content of this was trailed previously in speeches by Jo Johnson, and last week on the Conservative Home website. Plenty of  excellent commentary has already been provided through WonkHE and I’d encourage my colleagues  to read these as well as the paper itself.

Over on WonkHE I’d recommend the following:

David Kernohan writes on the  first TEF’sassessment

Martin McQuillan also focuses on TEF, and considers that universities brought TEF upon themsleves

Andrew McGettigan crucially shows ten things you might miss on a first reading

And in the Times Higher, Jo Williams questions the impact that TEF will have on how universities look at student satisfaction, and how this could affect academic freedoms.

There are some key points for us all to know about the contents of the Green Paper, specifically around the emerging detail of the Teaching Excellence Framework, but also the longer term ambitions for HE in this country. I’ve previously suggested in this blog that the changes are intensely political – they are about creating a deregulatory regime (although that doesn’t quite tally with the regulation that TEF might create), with opening up the “market” further to new entrants, and to allow more rapid shifts in market share. This seems to be driven by a belief that teaching in universities is poor (with little evidence of this) and that institutions need to be able to show better value for money to consumers. The assumption clearly is that HE does work as a market – whereas many including me, would argue that it is not a traditional market, that students are not passive consumers, that they should be active participants in creating their learning, and that universities themselves have created this dumbed down “the customer is right” approach.

Notable in the document is an emphasis on full time undergraduate students Part time barely gets a mention, and although there are nods of acknowledgement occasionally that the sector is more complex than FTUG, this barely affects proposals

Let’s look at the key proposals, on TEF in particular, and what they will mean for us in the short and medium term, remembering that this paper is now open for consultation.

Part A covers teaching excellence, quality and social mobility, and this is probably the section of most interest to most university teaching staff.

Excellence is defined in the paper as:

There is no one broadly accepted definition of “teaching excellence”. In practice it has many interpretations and there are likely to be different ways of measuring it. The Government does not intend to stifle innovation in the sector or restrict institutions’ freedom to choose what is in the best interests of their students. But we do think there is a need to provide greater clarity about what we are looking for and how we intend to measure it in relation to the TEF. Our thinking has been informed by the following principles:
• excellence must incorporate and reflect the diversity of the sector, disciplines and missions – not all students will achieve their best within the same model of teaching;
• excellence is the sum of many factors – focussing on metrics gives an overview, but not the whole picture;
• perceptions of excellence vary between students, institutions and employers;
• excellence is not something achieved easily or without focus, time, challenge and change.

Ultimately there will be different levels of teaching excellence – once the metrics and process have been decided –  however, in the first year, any institution that has a currently satisfactory QA review will qualify for level 1 TEF.

In the short term then, we, along wth nearly everyone else, would “pass” the TEF.

Later there will be up to 4 levels of TEF, and institutions will apply for the various levels. This will be done either on a rolling process or when unis decide that they are ready to apply for a higher level. This is where is starts to be contentious and competitive

A key driver of TEF is to link fees to teaching quality, and so in future the fee cap will likely be set by government for each of the different levels created, so that fees can once again be seen as a differentiator – the current system has led to nearly all traditional providers charging the maximum of £9000, so this has not helped generate the anticipated market behaviours. Whether having 4 values of fees will have any more effect remains to be seen.

The paper proposes that the metrics to be used  initially to be: DLHE, NSS and HESA continuation and retention. You might as well read a league table!

However, importantly, the paper recognises the limitations of metrics alone:

“However, we recognise that these metrics are largely proxies rather than direct measures of quality and learning gain and there are issues around how robust they are. To balance this we propose that the TEF assessment will consider institutional evidence, setting out their evidence for their excellent teaching.”

The institutional evidence might look a lot like the self evaluation submissions for quality activities, In addition to this, there is an expectation of using further new common metrics on engagement with study (including teaching intensity) and learning gain.

Degree classifications and grade inflation cited as a concern, with a nudge towards a move to GPA, but without explicitly asking universities to do so. The numbers of good degrees award nationally has risen dramatically over the last 10 years – partly because of good teaching, partly through a different student attitude to study and the need for a higher classification, but maybe partly through gaming the algorithms to gain league table position

On social mobility, the attainment gap between BME and white students is highlighted, as well as the lack of engagement with disadvantaged white males. These are areas that would be expected to be addressed in the TEF – “Work to improve access and success should have close links with the TEF.”

Other key headlines to be aware are:

  • removal of barriers to entry for new providers, including a possible remval of teh minimum numbers of students.
  • creation of an Office for Students – as part of a bonfire of quangos the work of HEFCE and OFFA would move a new OfS. The OfS would be responsible for assessing quality of teaching through TEF
  • proposed  to constitutional arrangements of Higher Education Corporations which will affect us
  • reducing complexity of research funding, although this will be detailed more in the coming Nurse review

So what do we need to do?

Firstly, a lot of people (everyone who works in the university) should read the Green Paper. This is clearly setting out this government’s ambitions for the future of the sector.

Secondly, a robust response to the consultation is needed – where there are sector concerns, for instance over the detail of TEF, we need to echo and articulate those.

We also need to accept that this is coming our way, so we can start preparing in anticipation now.

Ideally, we should be creating systems and processes for capturing information both qualitative and quantitative, as part of our business operating model, so that we don’t have to do this as a separate later exercise, but build it into standard practice (at the same time as remving other less useful and time consuming processes).

Since we know the metrics to be used in the second year, then we can start benchmarking ourselves against other universities using the data sets in Heidi. We can also do the same with the Unistats data

We can start amending our own portfolio performance review tools to reflect the same information but at subject level. This could be considered gaming the system – but we all know that’s one of the contributory factors in winning in the league table game, and this will be no different.

We can set up course monitoring processes such that detailed data and analysis is readily available to support the work we do on supporting students from widening participation and BME backgrounds – we know we work well in this area, but will want quantitative as well as quantitative evidence

We can start to articulate what does make for excellent teaching so that we are ready to develop our institutional qualitative submission – maybe a review of our ongoing course approval and review process can be carried out to start to capture this now.

Overall, a lot of work needs to be done, firstly to understand the devil in the details of the green paper, to identify how to set up processes and practices to deliver the information needed. The agile organisation will be working out how to do this without creating any extra burden, by removing other activities that are no longer necessary, and making sure that academic staff have the space to focus on the most important work – teaching, and scholarship and research that enables them to be excellent teachers

Most importantly though, is the need for us to keep demonstrating that excellence is not easily measured; that education is not just a product for consumption; that higher education is the  opportunity for people to engage in transformational activity which will provide benefits to themselves as individuals, as well as to broader liberal society.

 

 

 

 

Who are you?

Not the chant at a football ground, but a question we could be asking of ourselves – how well do we know our students?

At an institutional level, or indeed course level we can do two  things. Firstly we could carry out market segmentation that might tell us so much – games designers like to play computer games in their free time for instance. Secondly we can realise benefits through a data-centric approach to supporting teaching. If we can identify for instance, that for a given module, more than 50% of the students taking it entered university with BTEC qualifications, then maybe there are some conclusions for the lecturer to draw in terns of how to deliver and in particular how to assess. As we move into the era of TEF, then this ability to use data to measure outcomes might be come ever more important, with the comments made recently by Jo Johnson on the differential performance of students from WP or BME backgrounds

In terms of having a holistic view of who we teach, are there some generalities we can consider to give a holistic view, and will this help us to understand some of the current trends in higher education?

The dominant narrative in HE is always about 18-21 year-olds who are studying full time. These are the students who, for most institutions, generate most of the income, and also provide the outcomes that are subsequently used in league tables and are most likely to be used in the Teaching Excellence Framework. While recognising the importance of other groups of students, this article focuses on this grouping, since their behaviours tend to dominate policy.

Firstly we can think of how this generation of students can be defined in terms of their attitudes to technology. Born in 1997, they have never used a computer that was not controlled with a mouse or a touch interface. They have been used to having a mobile device all their lives, and a tablet computer for the last 5. So through senior school and public exams, they have been used to having easy access to the web. They can’t even envisage a time when the web wasn’t ubiquitous. They operate in a  world defined by online social media networks and physical storage is a strange concept. Clearly for us to be able to work with these students, we need to have our own high levels of digital capability, and this is a topic I will be returning to in coming weeks.

As well as understanding the technology expectations that these students bring to university, we can also look at some of their motivations. Focusing on young full time undergraduates, then they are what was termed in today’s Observer as  Generation K, or Katniss. (You may need to have teenaged children to understand this reference.)

…the economist and academic Noreena Hertz, who coined the term Generation K (after Katniss) for those born between 1995 and 2002, says that this is a generation riddled with anxiety, distrustful of traditional institutions from government to marriage, and, “like their heroine Katniss Everdeen, [imbued with] a strong sense of what is right and fair”

This generation worry about getting a job and this is understandable after major economic downturn, but the impact it might be having on these students’ attitude to higher education education is that they perceive HE as a means to getting a job. Although this is understandable, within the academy we still have a responsibility to show that higher education is more than that.

Writing in the same newspaper, WiIll Hutton, author of “The State We’re In” and now Master of Hertford College, Oxford, tackles the issues of the freedom to argue and the freedom to be challenged within universities. Hutton is primarily concerned with the retreat of liberalism and the diminution in importance of the idea of a public realm:

This disdain for notions of publicness has created a vacuum occupied by the rise of a libertarian individualism that indulges belief over reason. Non-falsifiable belief systems used only to be the hallmark of ideological communism or religious zealotry. Now, ideas, especially on the right but also to a degree on the left are less and less tested in a public realm by debate, with evidence marshalled to justify them. Instead, they are asserted as valid because the holder believesthem.

This new individualism, alongside the decline of the public, has provoked a mounting tide of, at best, siloed thinking, impervious to criticism, and, at worst, the indulgence of rank prejudice. Thus, on the right, if I feel that Britain is being swamped by immigrants, climate change is bogus, maleness is being overwhelmed by “femininazi” women or the welfare system is transfixed by cheating, then, whatever the facts, my feeling is valid because I hold it. That suffices without proof or evidence. In any case, there will always be some article in a rightwing paper to justify it.

Again, this individualism can be expressed by students – being risk-averse and not willing to be challenged or taken out of their comfort zones leads to behaviours that create “safe spaces” on campus, where we can no longer  hear certain views in case others are offended. In the same way, we might have a group of students who don’t want to be challenged too much in their learning, seeing a degree as a commercial transaction, rather than a transformational experience, the benefits of which may only be realised many years after completion. We ask students every year in the National Student Survey whether they felt their course was intellectually stimulating. We want our students to engage with their discipline, not as passive recipients, but as active scholars providing contribution and challenge. To do this though, we will need to remind ourselves of the tendency to be risk-averse, to want to be treated as a consumer and to be challenged just enough – but not so much it becomes too hard. We would do well to remember that as well as being stimulating, courses are supposed to challenge, be intellectually difficult  and to provide arguments that question individual beliefs.

Finally, this generation of students perceive themselves to be consumers. As argued in her book “Consuming Higher Education”, Joanna Williams of University of Kent argues that this attitude to consumption doesn’t come about just because of the introduction of tuition fees. Instead, Williams argues that we have a generation who have been led into this interpretation of what higher education is about, in part by the behaviours of universities:

“Rather than universities challenging the idea that a degree is an entitlement, institutions instead strengthen this notion. The provision of quantifiable information on contact hours assessment patterns and employment prospects suggests students are correct to to perceive of a degree as a product.”

Individual institutions can’t turn around and say we are having no part of this agenda as we are obliged to publish this information and no doubt will provide more in future. What we can do is explain better what the quantifiable data actually means, and more importantly, what it does not mean.

As we all write our responses to consultations on quality assurance, on the teaching excellence framework, and as we redevelop our own strategy documents in the light of a forthcoming Green Paper, we need to remind ourselves that universities do exist as more than degree factories that are there to produce “satisfied” customers. Understanding more about the backgrounds of our students and recognising the reasons for their consumerist behaviours may help us in the long term to be able to better articulate what the wider benefits of HE are, and the role of universities in liberal society.