End of Green Paper Consultation.

So after a few  months where policy wonks have pored over the fine details of the Green Paper consultation, on Friday the music stopped, and everyone had to submit their reponses. (actually the music stopped on the 10th).

Interestingly, there seems to be a high degree of consensus for once, between the various mission groups and representative bodies, particularity in relation to the proposed Teaching Excellence Framework.

Despite comments from Jo Johnson prior to release of the consultation, that “teaching was lamentable”, this view of a dysfunctional sector is not recognised by pretty much everyone else in it, and so such a toxic comment might be more the view of an individual, or small number of individuals who can see fault in one small part of the sector, rather than take a holistic view to see what those problems are in context.

The general response is that groups and individual institutions welcome the Green Paper and the commitment to supporting excellence in both teaching and research

Universities UK

The UUK response states that UUK:

  •  commits to working with the government to develop an effective Teaching Excellence Framework.
  • recommends that the second proposed iteration of the TEF be a pilot that gathers evidence on implementing teaching excellence
  • evaluates how this information can be usefully presented to students
  • disagrees with proposals for linking fee caps to multiple levels of TEF
  • proposes that the Office for Students should be called the Office for Students and Higher Education to reflect a broader mandate covering teaching funding, overview of research and third-stream activities.
  • Degree awarding powers should be based on four year track record and there should be a public interest test for granting university title
  • raises concerns about moving quality-related research funding (QR) into Research UK and asks for a clear commitment to protect dual support and the distinctiveness of the funding councils and Innovate UK

GuildHE

The GuildHE response again “welcomes the government’s focus on raising teaching standards and improving access to higher education.” and recognises that TEF “provides a real opportunity to enhance teaching within universities and raise the profile of good teachers but there are clearly many potential pitfalls along the way that we need to avoid”.

The problems around using existing metrics are articulated and:

This points to an evolution of TEF as something driven by the conversation around the teaching and learning opportunities that are delivered to students, i.e. TEF should become less about the metrics than about this conversation. There are very interesting parallels here to the current method of quality assessment – also configured as a conversation. Synergies between the new TEF intentions and the existing QA process should be identified and harnessed

Million+

Our own mission group, million+ has responded, “highlighting concerns in a number of areas and urging the Government to work with the sector to ensure that any changes introduced are in the interest of students, universities and employers” and again identifies problems with the existing metrics to populate TEF:

“Linking fee increases with a Teaching Excellence Framework (TEF) based on metrics that are proxies for teaching quality, is unlikely to provide students or employers with an accurate picture of the rich and varied teaching and learning environments that universities offer. This risks damaging the reputation of the higher education sector in the UK and is why we recommend that the government defer the introduction of a multi-level TEF in 2018 until further work has been undertaken to determine the best way to promote teaching excellence.”

CDBU

Finally to be covered here, the Council for the Defence of British Universities has responded, and not surprisingly, is not supportive of much of the Green Paper:

the Teaching Excellence Framework (TEF) – is put forward to address putative problems, without providing any evidence that these problems exist. The proposed TEF would be an expensive and bureaucratic system that would entail increasing complexity and disruption for years to come. The use of proxies, such as the NSS or graduate income, for teaching excellence is at odds with the ethos and values of education and scholarship. Both the content and the methodology behind the Green Paper come across as counter to the academic values that lie at the heart of any university worthy of the name. These values include reliance on reason, argument, and evidence; critical and creative thinking; rigorous analysis of data; and precise and meaningful communication. There is no recognition in the Green Paper that the primary purpose of universities is to foster these values; instead, universities are equated with businesses, value is defined purely in economic terms, and students and staff are set up in opposition as consumer and vendor respectively, working to serve conflicting interests (to pay as little as possible for the product purchased and to charge as much as the ‘customer’ will take).

 

For those who would like a critical nuanced exposition of the Gren Paper, then I’d recommend the ever excellent Stefan Collini, writing in the London Review of Books with an article titled “Who are the spongers now?”, where on TEF he concludes:

“So what will the TEF actually produce? At a minimum, the following: more administrators to administer the TEF; a greater role for business in shaping the curriculum and forms of teaching; a mountain of prose in which institutions describe, in the prescribed terms, how wonderful their provision and procedures are. It also seems pretty certain to produce more efforts by universities to make sure their NSS scores look good; more pressure on academics to do whatever it takes to improve their institution’s overall TEF rating; and more league tables, more gaming of the system, and more disingenuous boasting by universities about being in the ‘top ten’ for this or that.

What is it unlikely to produce? Better quality teaching”

So we seem to have stakeholders from across the sector, saying pretty much some of the same things:

  • the metrics proposed as proxies for teaching excellence, can’t be relied upon to show teaching excellence
  • changes to the routes for new providers are being regarded with some suspicion
  • the time allowed to develop and implement TEF is too short
  • there is concern about linking fees to varying levels of TEF
  • dual support of research should be supported

What will be interesting now is to see how government responds to a sector who know themselves well, and who have identified where more work is needed to translate policy proposals into actions – with this overwhelming amount of agreement, it will be hard to push ahead exactly as outlined in the Green Paper. Lets’s leave the last words to Collini again:

“But don’t worry: the Green Paper is only a ‘consultation’ document. That must mean that if cogent objections are put forward to the premises, reasoning and conclusions it contains, none of these proposals will come to pass. Well, mustn’t it?”

Patterns and Trends in UK HE 2015

An annual report from Universities UK looks at trends in the UK HE sector based primarily on statistics from HESA, and looking over a 10 year period.

This is a period that has seen significant change, and when we look at the proposals in the recent Green Paper, there is little to suggest that the pace of change will be slowing.

A quick review of the report then gives us some business insight in to the sector generically, from which we can extrapolate for our own institution.

The three chapters consider: students, staff and finances. This blog piece will focus on students

Under students we can see the changes in make-up of the student population – full time numbers, both postgraduate and undergraduate, show an increase, while part time study clearly has declined.

uuk1

The contiued and drastic decline in PTUG is explained by

the removal in 2008–09 of funding for students taking qualifications equivalent to or lower than ones which they already had, and by reforms to undergraduate funding in 2012–13, including an increase in fees following cuts to teaching grants and issues around eligibility for tuition fee loans. At the same time the economic downturn has also caused a reduction in the number of students able to self-fund part-time study, and a reduction in the number of employers willing to support employees through part-time study.

The suggestion in eh recent Green Paper on removing ELQ restrictions for thise wishing to study STEM subejcts may help slightly in this area, but the contiued decline in part time numebrs alsoe reduces the chances of universties being able to tackle issues of social mobility, again a key part of the Green Paper.

While undergraduate numbers overall have increased, this increase is due to the numbers studying for a first degree. At the same time, the numbers enroling into other undergraduate courses has dropped significantly:

uuk2

Which would seem to have some pretty strong messages abut the worth of developing a number of courses!

On international student numbers, despite growth in the numbers of students who travel to study, there has bee a decline in the UK of international students, as we lose out to other markets such as USA and Australia.

uuk3

We may be seeing the impact of the UK view of students being part of the net migration target and the removal of PSW visas. As UUK highlight:

the decline in students from India are fuelling concern about the UK’s ability to attract international students, which could damage the higher education sector.
International student fee income accounted for 13% of sector income in 2013–14, and demand from international students can support the provision of certain strategically important subjects in the UK (eg engineering, technology and computer science, particularly at postgraduate level where around half of all students are from outside the EU). Higher education is important to the wider UK economy, with non-EU students contributing £7.2 billion to the UK economy and supporting around 137,000 jobs through their tuition fees, accommodation and off-campus expenditure in 2011–12.

Looking at subject of study over the 10 year period shows some clear trends, much of which is backed up by our own detailed analysis of UCAS data.

uuk4

The most surprising part of this is perhaps the overall drop in computer science, at the same time as students with god computer science skills are desperately needed by employers. The huge drop in subjects described as “combined” should not come as a shock – maybe students are looking for courses that they believe gofer a clear route to employability? Interesting, Keele, our nearest geographical competitor, is now developing courses in Liberal Arts and Natural Sciences, which initially might seem to fly in the face of this trend.

The report notes the number of people in the workforce who now have tertiary level skills, but recognises that the UK lags behind other countries in this regard. Also, looking to the future, a clear need is identified for an increased number of people with undergraduate and postgraduate skills for the needs of the international workforce of the future.

uuk5

This rise in demand for those with postgraduate qualifications ins a clear opportunity, and the recent Green Paper proposals for plans to make loans available to students wishing to enter PG study provides a clear opportunity.

The paper ends by summarising changes in student numbers, income and staffing:uuk6

and concludes:

Despite the challenges of a dynamic and sometimes uncertain operating environment, the UK higher education sector can point to a number of successes over the period……In this increasingly competitive international environment and with a referendum on remaining in the EU taking place before the end of 2017, the UK higher education sector will continue to work hard to maintain its position and attract the staff, students, funding and partnerships that are central to its success.

 

 

Presentation to Academic Group Leaders

We regularly hold a forum at Staffordshire University for our Academic Group Leaders – these are the senior academic staff who are responsible for line managing and leading groups of academic colleagues.

This week I led the forum, with a presentation on league tables and on some of the implications of the recent Green Paper “Fulfilling our Potential. Teaching Excellence, Social Mobility and Student Choice“.

The slides can be viewed here at Slideshare

 

Wonk’s Delight

And so the waiting is over. Friday finally saw the long awaited release of one the  most anticipated artifacts of 2016. Twitter was inundated with commentary, and major newspapers rushed to add reports to their online editions as the new John Lewis Christmas advert became available.

In other news, the government published its green paper – “Fulfilling out Potential – Teaching Excellence, Social Mobility and Student Choice”.

Much of the content of this was trailed previously in speeches by Jo Johnson, and last week on the Conservative Home website. Plenty of  excellent commentary has already been provided through WonkHE and I’d encourage my colleagues  to read these as well as the paper itself.

Over on WonkHE I’d recommend the following:

David Kernohan writes on the  first TEF’sassessment

Martin McQuillan also focuses on TEF, and considers that universities brought TEF upon themsleves

Andrew McGettigan crucially shows ten things you might miss on a first reading

And in the Times Higher, Jo Williams questions the impact that TEF will have on how universities look at student satisfaction, and how this could affect academic freedoms.

There are some key points for us all to know about the contents of the Green Paper, specifically around the emerging detail of the Teaching Excellence Framework, but also the longer term ambitions for HE in this country. I’ve previously suggested in this blog that the changes are intensely political – they are about creating a deregulatory regime (although that doesn’t quite tally with the regulation that TEF might create), with opening up the “market” further to new entrants, and to allow more rapid shifts in market share. This seems to be driven by a belief that teaching in universities is poor (with little evidence of this) and that institutions need to be able to show better value for money to consumers. The assumption clearly is that HE does work as a market – whereas many including me, would argue that it is not a traditional market, that students are not passive consumers, that they should be active participants in creating their learning, and that universities themselves have created this dumbed down “the customer is right” approach.

Notable in the document is an emphasis on full time undergraduate students Part time barely gets a mention, and although there are nods of acknowledgement occasionally that the sector is more complex than FTUG, this barely affects proposals

Let’s look at the key proposals, on TEF in particular, and what they will mean for us in the short and medium term, remembering that this paper is now open for consultation.

Part A covers teaching excellence, quality and social mobility, and this is probably the section of most interest to most university teaching staff.

Excellence is defined in the paper as:

There is no one broadly accepted definition of “teaching excellence”. In practice it has many interpretations and there are likely to be different ways of measuring it. The Government does not intend to stifle innovation in the sector or restrict institutions’ freedom to choose what is in the best interests of their students. But we do think there is a need to provide greater clarity about what we are looking for and how we intend to measure it in relation to the TEF. Our thinking has been informed by the following principles:
• excellence must incorporate and reflect the diversity of the sector, disciplines and missions – not all students will achieve their best within the same model of teaching;
• excellence is the sum of many factors – focussing on metrics gives an overview, but not the whole picture;
• perceptions of excellence vary between students, institutions and employers;
• excellence is not something achieved easily or without focus, time, challenge and change.

Ultimately there will be different levels of teaching excellence – once the metrics and process have been decided –  however, in the first year, any institution that has a currently satisfactory QA review will qualify for level 1 TEF.

In the short term then, we, along wth nearly everyone else, would “pass” the TEF.

Later there will be up to 4 levels of TEF, and institutions will apply for the various levels. This will be done either on a rolling process or when unis decide that they are ready to apply for a higher level. This is where is starts to be contentious and competitive

A key driver of TEF is to link fees to teaching quality, and so in future the fee cap will likely be set by government for each of the different levels created, so that fees can once again be seen as a differentiator – the current system has led to nearly all traditional providers charging the maximum of £9000, so this has not helped generate the anticipated market behaviours. Whether having 4 values of fees will have any more effect remains to be seen.

The paper proposes that the metrics to be used  initially to be: DLHE, NSS and HESA continuation and retention. You might as well read a league table!

However, importantly, the paper recognises the limitations of metrics alone:

“However, we recognise that these metrics are largely proxies rather than direct measures of quality and learning gain and there are issues around how robust they are. To balance this we propose that the TEF assessment will consider institutional evidence, setting out their evidence for their excellent teaching.”

The institutional evidence might look a lot like the self evaluation submissions for quality activities, In addition to this, there is an expectation of using further new common metrics on engagement with study (including teaching intensity) and learning gain.

Degree classifications and grade inflation cited as a concern, with a nudge towards a move to GPA, but without explicitly asking universities to do so. The numbers of good degrees award nationally has risen dramatically over the last 10 years – partly because of good teaching, partly through a different student attitude to study and the need for a higher classification, but maybe partly through gaming the algorithms to gain league table position

On social mobility, the attainment gap between BME and white students is highlighted, as well as the lack of engagement with disadvantaged white males. These are areas that would be expected to be addressed in the TEF – “Work to improve access and success should have close links with the TEF.”

Other key headlines to be aware are:

  • removal of barriers to entry for new providers, including a possible remval of teh minimum numbers of students.
  • creation of an Office for Students – as part of a bonfire of quangos the work of HEFCE and OFFA would move a new OfS. The OfS would be responsible for assessing quality of teaching through TEF
  • proposed  to constitutional arrangements of Higher Education Corporations which will affect us
  • reducing complexity of research funding, although this will be detailed more in the coming Nurse review

So what do we need to do?

Firstly, a lot of people (everyone who works in the university) should read the Green Paper. This is clearly setting out this government’s ambitions for the future of the sector.

Secondly, a robust response to the consultation is needed – where there are sector concerns, for instance over the detail of TEF, we need to echo and articulate those.

We also need to accept that this is coming our way, so we can start preparing in anticipation now.

Ideally, we should be creating systems and processes for capturing information both qualitative and quantitative, as part of our business operating model, so that we don’t have to do this as a separate later exercise, but build it into standard practice (at the same time as remving other less useful and time consuming processes).

Since we know the metrics to be used in the second year, then we can start benchmarking ourselves against other universities using the data sets in Heidi. We can also do the same with the Unistats data

We can start amending our own portfolio performance review tools to reflect the same information but at subject level. This could be considered gaming the system – but we all know that’s one of the contributory factors in winning in the league table game, and this will be no different.

We can set up course monitoring processes such that detailed data and analysis is readily available to support the work we do on supporting students from widening participation and BME backgrounds – we know we work well in this area, but will want quantitative as well as quantitative evidence

We can start to articulate what does make for excellent teaching so that we are ready to develop our institutional qualitative submission – maybe a review of our ongoing course approval and review process can be carried out to start to capture this now.

Overall, a lot of work needs to be done, firstly to understand the devil in the details of the green paper, to identify how to set up processes and practices to deliver the information needed. The agile organisation will be working out how to do this without creating any extra burden, by removing other activities that are no longer necessary, and making sure that academic staff have the space to focus on the most important work – teaching, and scholarship and research that enables them to be excellent teachers

Most importantly though, is the need for us to keep demonstrating that excellence is not easily measured; that education is not just a product for consumption; that higher education is the  opportunity for people to engage in transformational activity which will provide benefits to themselves as individuals, as well as to broader liberal society.

 

 

 

 

Politics and the TEF

Prior to the general election, I wrote a blog post reviewing the various parties’ views on HE. Following the conservative majority I wrote another piece which concluded with “What is still not clear is how universities might be regulated, how quality mechanisms will operate in future, and how the regulatory and quality regime will be changed to encompass the more diverse range of providers”

Following the various party conferences we now enter a period when we await, with bated breath, the green paper on higher education. For an insight into the Conservative conference, then I recommend “Welcome to the Northern Powerhouse of Cards” by Martin McQuillan of Kingston University

There’s little point in looking at the other parties right now – there is not likely to be an election till 2020, and Labour haven’t identified their position on fees, let alone how they will carry out the role of opposition to the green paper.

The Conservatives are in an interesting situation. Cameron as leader, who has acted as a CEO has already indicated his intention to step down. Hence for everyone else it “eyes on the prize”. As deputy CEO, Osborne has been calling the shots on HE policy, since the Treasury is dictating policy more clearly than any other department. May is setting out her stall, and showing clear opposition to overseas students which will win her no friends in universities. Boris is harrumphing around the margins, and looking more widely Hunt is exerting everyone to work harder.Meanwhile, Javid is happy to drive through large cuts at BIS, and we can expect that many of the organisations that currently work in the HE sector may cease to exist.

It’s into this environment, with his boss supporting 40% cuts to BIS, that Johnson will need to produce  a green paper and ultimately drive legislation through parliament

All of a sudden,this looks threatening to HEFCE. The HEFCE consultation on QA is in tune with government and seems to promote a move to a deregulatory ideology and imply the demise of QAA. More recently though, with questions being asked about whether the remaining amounts of funding could be administered from elsewhere, and the need for a body to run TEF, then HEFCE themselves look more vulnerable.

The Teaching Excellence Framework will clearly be a big part of the green paper. It was a commitment from Osborne (that Treasury driver again) and is detailed in the government’s productivity plan “Fixing the foundations:Creating a more prosperous nation”

Excellence in teaching
4.7 The government will introduce a new Teaching Excellence Framework to sharpen incentives for institutions to provide excellent teaching, as currently exist for research. This will improve the value for money and return on investment for both students and the government, and will contribute to aligning graduate skills and expectations with the needs of employers. The government will consult later this year on how a Teaching Excellence Framework can be developed, including outcome-focussed criteria and metrics. The Teaching Excellence Framework will inform student decision-making, continue to support a high average wage premium for graduates and ensure that students’ hard-won qualifications keep their value over time.
4.8 To support teaching excellence, the government will allow institutions offering high quality teaching to increase their tuition fees in line with inflation from 2017-18, and will consult on the mechanisms to do this. This will reward excellent institutions with higher fee income, while ensuring students get good value from the tuition loans that the government underwrites.

Johnson now needs to steer this through parliament, at the same time as BIS is facing large cuts, and he needs to produce something that will work, both as a fix in the short term, and as a longer term evaluation of teaching.

To be able to have variable fees from 2017-18 will mean measures in place during the current academic year. Inevitably this will be based on existing measures – NSS, Hesa returns, DLHE initially.

Longer term though, then a new set of measures will come in which will provide challenges to the sector, and to individual institutions. From the Times Higher Johnson has made it clear how he would like the metrics to be set up:

Widening participation and access will be intimately linked to the TEF. One of the core metrics we envisage using in the TEF will be the progress and the value add [for] students from disadvantaged backgrounds, measuring it for example in terms of their retention and completion rates. And their [universities’] success in moving students on to either further study or graduate work.

On having an impact on further marketisation, then Johnson says:

the system should “not only have the capacity for more rapid market entry, but we [should] have the capacity for more rapid market share shifts between universities than we have hitherto seen in the sector”.

and  that

he wanted a system where “market share can shift towards where teaching quality really resides. Our teaching excellence framework will be an important signal to students of where quality resides, discipline by discipline, institution by institution.”

He’s asking an awful lot from a set of metrics that are not yet defined, and that will have numerous questions raised by many in the sector.

In the meantime, what can individuals and institutions do?

Firstly there is the opportunity to respond to the government’s inquiry into assessing the quality of HE, which asks specific questions such as:

  • .What should be the objectives of a Teaching Excellence Framework (‘TEF’)?
  • What are the institutional behaviours a TEF should drive? How can a system be designed to avoid unintended consequences?
  • What should be the relationship between the TEF and fee level?

Secondly we can  start looking at the various measures of value added or learning gain for different groups of students. HEFCE are already supporting a range of projects involving over 70 institutions to look at learning gain.

One of the unintended consequences that TEF might bring about is a gaming of the system. I’m not suggesting that data returns that feed into league tables are inaccurate, but one part of a successful league table result is a set of carefully constructed data returns. It’s equally likely that it will be possible to do something similar with any TEF submission, so all institutions will learn very quickly how to report data in the best possible way

Finally, recognising that TEF will be used to drive rapid shifts in market share (a euphemism?) then we will all need to get very good, not only at supporting the widest range of students, but also at understanding how the metrics apply to us, and how we can build internal systems to replicate them.

 

 

 

Jo Johnson’s Speech to UUK

For the second time, our Minister with responsibility for universities makes a speech on HE, this time to Universities UK and an assembled throng of Vice Chancellors. Our own VC’s impressions will be in his blog next week.

The full text of the speech is here, Higher education: fulfilling our potential, but are there any hints of what is to come?

there is considerable unfinished business and the green paper will seek views on the changes the government believes will be necessary to ensure that higher education continues to be a great national success story in the years to come

We can expect to see the Green Paper that will address this unfinished business in the current session of parliament.

The next section of the speech is about “teaching at the heart of the system”, which is different from the previous mantra of “students at the heart of the system” which has subsequently become a strapline in many a university mission statement.

At the centre of this vision are the young people contemplating their futures in a world where no one owes them a living, where they must depend on their wits and drive to survive.

Well-equipped students ready to contribute to society and to businesses keen to employ increasing numbers of skilled graduates

So once again is higher education being seen as a transactional good, rather than a transformational experience, just for young people to enter the employment market, red in tooth and claw? Even though a majority of students may be young, we need to remind ourselves that HE is not just for young people, and not just to deliver training for employment.

Moving on to talk about the proposed Teaching Excellence Framework (however with no proposals or suggestions at this stage), Johnson refers to teaching staff who go the extra mile providing feedback and email replies at weekends, as well as those who think:

we’ll award you the degree as the hoped-for job ticket in return for compliance with minimal academic requirements and due receipt of fees’

I’m sure plenty will take issue with this – although in a sector that employs over 194,000  academic staff (HESA data 2013-14), there must be a few individuals in this category although we wouldn’t expect an institution to behave overall in this way. The speech goes on to talk about the variability of student experience and:

There is extraordinary teaching that deserves greater recognition. And there is lamentable teaching that must be driven out of our system. It damages the reputation of UK higher education and I am determined to address it.

Is the comment on lamentable teaching about individuals or about entire institutions? The TEF is  to  assess an institution, not individual performances. The answer to that might provide an answer of whether solutions are to be centrally driven or can be dealt with through internal management processes and enhancement activities.

Johnson suggests that more information will be provided in future about kind of teaching that students will receive. Having been to a number of open days in other institutions recently I’v e been struck by the willingness to brag about high contact hours, with little reference to what they actually involve.

The new framework will “reward universities that do most to stretch young – and also not so young – minds”. At last a recognition that not all students are 18 years old – but the devil will be in the detail of this: how will the framework actually measure the amount of stretch? This may lead us to look at ideas of learning gain and value-added, but can this be put in place quickly enough for TEF to be ready for 2017?

Johnson asks for there to be  a shift in how we think about teaching – and in an institution that is focused more on teaching than research, then this should be good news.

Widening Participation appears as a major part of the speech – and at a time when cuts to the BIS budget are due, and many commentators suggesting that the Student Opportunity Fund which supports WP activities is likely to be under threat, then we have a series of laudable ideas but which will need money to support.

There is a clear commitment to the use of HE as a driver of social mobility with particular emphasis given to participation rates of working class white boys, and students from BME backgrounds.

Plenty of reference is made to the data provided by UCAS, which will provide the necessary information on recruitment into universities, and a comment is made about linking this to other datasets. If we are serious about being able to understand how well WP activities are working, then we need to look not just at UCAS and recruitment data, but also at information on retention, progression and achievement – only when we can see significant reductions in the differences between different groups in all of the relevant factors would we be able to say that we are making steps to increase social mobility. Much of this information is already held within universities, and plenty of them provide analysis of this to explain their outcomes.

On alternative providers, Johnson recognises that not everyone wants to study a 3 year degree, and praises the alternative provider sector in being able to offer other provision, through validation arrangements with existing universities, but suggests that the current process “stifles competition, innovation and student choice, which is why we will consult on alternative options for new providers if they do not want to go down the current validation route.”

On regulation, Johnson states that this a a deregulatory government, and the recent consultation on QA from HEFCE seemed to push towards a system of no central agency, but with increased role of governance (similar to the rise in the school sector of  Academies and removal of Local Authority control?).

So we need a simpler, less bureaucratic and less expensive system of regulation. A system that explicitly champions the student, employer and taxpayer interest in ensuring value for their investment in education and requires transparency from providers so that they can be held accountable for it. One that protects institutional autonomy and academic freedom and maintains the highest quality of higher education, safeguarding the strong international reputation of English universities

And this is clealry talking about the role of HEFCE rather than universities in countries with devolved administrations.

Overall – will this please the sector?

  • A commitment to great teaching won’t be argued with – the mechanisms of assessing it will be.
  • The change in regulation for alternative providers might be seen as a threat to some institutions (probably only those in the bottom quartile of league tables, or current FE providers of HE)
  • The focus on widening participation should be welcomed – provided that funding and full data analysis is part of the deal.

As ever, the details have yet to merge and plenty of other commentators will have more to say on this speech than I, and we will all eagerly await the Green Paper, and in particular the plans for a Teaching Excellence Framework and a renewed focus on widening participation.

 

Do the numbers matter?

We are now at the point in the year where we start getting hold of course level metrics – from employability through DLHE, for student experience from NSS and on student performance in terms of retention and attainment through our own datasets.

Bringing these together means that we can create a snapshot of how “well” a course might have performed in the last years.

There have been a number of publications over the summer on the use of numbers and metrics, in particular the report “The Metric Tide” which reflects in the use of metrics to assess research excellence.

However this publication also contains chapters on management by metrics and on the culture of counting, and as someone who works extensively on looking at the performance of our portfolio of courses, as well as league tables, this was of interest.

“Across the higher education sector, quantitative data is now used far more widely as a management aid, reflecting developments in the private sector over recent decades. ……………….., most universities now plan resource allocation centrally, often drawing on the advice of dedicated intelligence and analysis units that gather information from departments and faculties. The use of such systems has helped universities to strengthen their reputation as responsible, well-managed institutions. The relatively robust financial position of the sector, and the continued trust placed in universities by public funders to manage their own affairs, is in part founded on such perceptions of sound financial governance.

The extent to which management systems in HEIs help or hinder institutional success is of course contested. On the positive side, such systems have helped to make decision making fairer and more transparent, and allowed institutions to tackle genuine cases of underperformance. At the same time, many within academia resist moves towards greater quantification of performance management on the grounds that these will erode academic freedoms and the traditional values of universities. There is of course a proper place for competition in academic life, but there are also growing concerns about an expansion in the number and reach of managers, and the distortions that can be created by systems of institutionalized audit.”

 

What is important then is how we deal with  data. A list of numbers alone does not create useful management information. Indeed even a collation or aggregation of all the data (similar to a league table approach) still is only one part of the picture.

What data or information such as this does provide us with, are some insights into how different parts of the university are faring, or how our different groups of students see us.

The useful work starts when we realise how to use the numbers – this is where we now have those conversations with course teams to find out why a metric is particularly high or low. Is there some really great practice that can be shared with other people? Is there a reason for a disappointing NSS score?

Only by going beyond the numbers and engaging with the course teams will we get the full insight into why the results are as they are.

This is not to say that everything can be explained away. The whole point of building up a metrics approach to assessing what we do is threefold:

  • To make sure all colleagues are aware of how measurable outcomes affect us reputationally and reflect the results and experience of actual students
  • To provide a consistent reliable management information to act as a trigger
  • To raise the data understanding capability of all groups of staff.

We should not be afraid of looking at metrics to judge a programme, but as well should become better at using that information  to be able to understand exactly why we perform that way.

As well as looking at the raw data, we also need to look closely at what it is we are trying to achieve, and how this might influence how we set up benchmarks and targets. Some examples might be:

  • Benchmarking NSS results for subjects against the sector average for that subject. This shows how well we do in comparison with others rather than a comparison against an internal university average score (guess what – half our courses were above average)
  • Considering a calculation of value added instead of good degree outcomes. For a university with a significant intake of widening participation students, this might be  a better reflection of “distance travelled” and show the results of our teaching. Any VA score should have to be different form that used in one of the league tables, which only considers 1sts and 2(i)s as a good outcome. For some students, a 2(ii) might be appropriate.

We should all be aware that using metrics to assess quality and performance is becoming increasingly important.

The current consultation from HEFCE on the future of quality assurance has a number of major themes, but two of these are around data and governance.

In the proposals are the suggestions that quality could be assured by a university identifying its own range of measures that indicate quality, and that governing bodies will be in a position to make judgements of success against these.

This could be an opportunity to create a set of metrics that really measure where we want our successes to be and that are actually aligned to the mission of the university., rather than the ones that might suit another university more readily.

Secondly, it does mean that governing bodies (and the people that brief them) will need to become more aware of data, its limitations and meanings.

Finally, and this is a concern – the proposed Teaching Excellence Framework will most likely be put in place very quickly, and will be metrics based. In the time available, this might only be based on metrics and measures that are already well known and used – NSS, DLHE, good degrees (not dissimilar to a league table so far). Since the ability to charge increased fees will depend on success in the TEF, then it does mean that despite in future possibly being able to identify what our measures of success will be, in the short term we cannot stop focussing on those key indicators.

 

Consultation on QA Arrangements

In case you missed it last week, HEFCE have now published their consultation into arrangement for quality assurance in higher education.

Much of what appears in the document was already trailed, perhaps what is most interesting is the reaction seen across the sector since publication.

The key themes as described by HEFCE are:

  • A shift from process-driven assurance to analysis of student academic outcomes. A number of respondents to the first phase of the review wished to see this shift. It builds on existing institutional activity to drive excellence and innovation in learning and teaching in the context of an institution’s own mission, location and modes of delivery, and the nature of their student body.
  • Strengthening the existing external examining system to protect the integrity of academic standards. There was strong support in the first phase of the review for the external examining system, but recognition of the need for further modernisation and professionalisation.
  • An enhanced role for universities’ and colleges’ own assurance systems. Governing bodies would confirm that their senates or academic boards were reviewing the quality of their students’ academic experience and (for institutions with degree awarding powers) academic output standards, and provide assurance that there were appropriate action plans in place where necessary

Reading through the document, then for me, three theme become of increasing importance:

  • the increasing role of university governance
  • the need for internal data to provie assurance
  • the development of a teaching excellence framework.

One organisation who don’t get a mention at all in the document is QAA. Their response included:

‘QAA can bring extensive expertise to this debate. We will be offering ideas to shape a genuinely risk-based, proportionate approach, tailored to the track record and circumstances of each individual college or university; an approach that is truly UK-wide and underpins the reputation of UK higher education internationally”

In a speech earlier in the week by the Chief Exec of QAA, Anthony McClaren, their view was made a little more forcefully:

There are also a number of fundamental principles missing from what is being proposed.

The value of external cyclical review and the critical role it has in protecting the interests of our students, supporting providers developmentally through enhancement, providing public assurance, complying with European standards and safeguarding the global reputation of the sector. And not, as suggested in the consultation, merely a ‘repeated retesting against baseline requirements’.

Nor does it properly recognise the importance of a coherent system with a single independent quality assurance body, a single body which avoids fragmentation and weakening of the system, and enables a level playing field covering not only publicly funded universities and colleges, but also alternative providers.

Also, the retention of a UK-wide system and, critically, a single UK-wide framework as we have today, which is respected and trusted globally.

And a system which continues to meet fully now – not as an aspiration for the future – both European and wider international expectations.

And with international quality assurance activities which continue to support UK providers both in recruiting international students to this country and with their transnational education activities overseas.

QAA, working with the UK sector, is known, trusted and respected round the world as the safeguarder of quality and standards in UK higher education. Given the international objectives of the sector and also our government’s export ambitions, our work will become even more crucial in the future.

We will be responding to this consultation.

 

Million+ responded to the consultation with a piece by it’s Chair, our VC Prof Michael Gunn:

 

The consultation raises a number of complex issues and universities will wish to carefully consider their responses. However if the end result is that England loses an independent external quality assurance system there would be concerns about the impact on the reputation of UK higher education both within the UK but also overseas.

Universities UK responded with:

Effective quality assessment will continue to play a central role in securing our global reputation and providing assurances to students, the government, and the public more widely. It is important that this remains fit for purpose for the whole of the United Kingdom and in a significantly changed higher education environment, adapting to increasing diversity in students and institutions. The proposals set out by the English, Welsh and Northern Irish funding bodies pick up this challenge, setting out clear proposals for reform

In response to the various voices making themselves hear, and in particular the fact that the original document does seem to have a few bits missing, then HEFCE provided a blog article entitled “No consultation document survives first contact with its stakeholders (without the need for further elaboration)”.

Here HEFCE say:

The consultation document’s first formal engagement with the world has revealed the need for further elaboration and explanation, but the proposals themselves are holding up.  And the purpose of the consultation is to set out proposals and then to gather and test responses.  And then to think some more

Clearly this is going to be a major piece of work through the summer, not just for HEFCE, but for all relevant stakeholders.

Areas that we might want to think about are how we involve governance more centrally in assuring standards, which links to how we provide information to allow such judgments to be made.

Finally this week the new universities minister, Jo Johnson, announced  plans to create a Teaching Excellence Framework. Clearly this is links to the HEFCE consultation, and will be the next challenge for us to face. Hopefully we won’t just be replacing one review of quality that focused on process rather than outcomes,  with another for teaching that focuses on process.

 

 

 

 

Jo Johnson – First Speech

Our new universities minister has broken cover and delivered his first speech about higher education, entitled “Teaching at the Heart of the System” to UUK this week.

Here’s the edited highlights:

On helping students to make informed choices:

we can now start to assess the employment and earnings returns to education by matching Department for Business, Innovation and Skills (BIS) and Department for Education (DfE) education data with HMRC employment and income data and Department for Work and Pensions (DWP) benefits data.

We will be able to see which institutions and subjects provide the greatest financial benefits to students, and reflect this back to potential applicants.

On value for money:

While independent learning is vital, universities must get used to providing clearer information about how many hours students will spend in lectures, seminars and tutorials, and who will deliver the teaching.

Indeed the Competition and Markets Authority have advised higher education providers that information should be available to prospective students to meet the requirements of consumer law.

I’ve been rewriting module handbook templates recently for one of our faculties, with a focus on managing student expectations, and clealry identifying how we deliver courses, how much independent learning is expected, and crucially, what we expect our students to do through independent study.

On employability:

Last year’s CBI/Pearson Education and Skills survey suggested that 47% of employers felt universities should do more to help students become job-ready.

Government, business and the university sector need to come together to address this mismatch between supply and demand in the graduate labour market.

Businesses should not just be seen as customers of universities, recruiting the graduates they educate or buying research expertise, but as active partners.

Although this seems to reinforce the narrative that universities are just in the business of providing work ready employees, we do recognise the importance of developing employability skills, hence our recent Learning and Teaching Conference that focused on this very topic.

The big announcement though is this one:

There must be recognition of excellent teaching – and clear incentives to make ‘good’ teaching even better.

Some rebalancing of the pull between teaching and research is undoubtedly required.

It is striking that while we have a set of measures to reward high quality research, backed by substantial funding (the Research Excellence Framework), there is nothing equivalent to drive up standards in teaching.

That is why my priority as Universities Minister will be to make sure students get the teaching they deserve and employers get graduates with the skills they need by introducing the Teaching Excellence Framework we promised in our manifesto.

While no one would argue that we shouldn’t have excellent teaching, the difficulty here will be in finding a way of assessing excellence without becoming overly prescriptive or burdensome (REF anyone?), although the minister does say that “any external review must be proportionate and light touch, not big, bossy and bureaucratic”. Interestingly, there is a hint of a future role for QAA in developing the framework (just as they may be losing the remit of institutional quality assurance).

On good degrees, Johnson notes the rise in the number of 1sts and 2(i)s being awarded, saying:

To the extent this expansion in the number of firsts and 2:1s is to do with rising levels of attainment and hard work, I applaud it.

But I suspect I am not alone in worrying that less benign forces are at work with the potential to damage the UK higher education brand.

On the face of it, the facts are certainly startling.

There has been a 300% increase in the percentage of firsts since the 1990s

Maybe the less benign forces come from the impact of league tables and the need to succeed in these to maintain institutional success? Alternativley, maybe universities have recognised how they can improve student attainment and success through the right kinds of interventions?

The proposed Teaching Excellence Framework will be expected to tackle degree classification inflation ( assuming that this does of course exist)

So far the speech has been cautious welcomed (TEF will be the biggest concern) with UUK saying:

Providing a high-quality, world-leading experience for all students is central to what our universities do, and they are always seeking to improve what they offer to students. We will be considering carefully how a new Teaching Excellence Framework can best add value to all students, whatever their choice of subject or university, and whatever their background and aspirations. The challenge is how to construct a single Framework that can effectively respond to that tremendous diversity. Universities UK will be contributing to the consultation process in the coming months

And from million+:

Professor Michael Gunn, chair of the university think-tank million+ and Vice-Chancellor of Staffordshire University said:

“Universities are engaged in high quality teaching and research but too often teaching has played second fiddle in discussions about the value and contribution of higher education to society and the economy.

“We warmly welcome Jo Johnson’s recognition that the student body is talented and diverse in background, age, mode of study and pre-entry qualifications.

“His commitment to work with employers to highlight shared responsibilities for graduate employability also opens up the potential to improve recruitment practices in ways that would benefit many students.

“The Minster has undoubtedly set a number of challenges and we welcome his commitment to consult widely and the opportunity to explore ways to value the excellent teaching and support for learning in universities.”

There will be a Green Paper in the autumn, so something else (as well as the HEFCE consultation on QA) for policy wonks to get excited about.

 

Why are universities important?

At the time when we wait with bated breath for the Chancellor to announce where cuts to the BIS budgets will come ,Universities UK has produced a timely publication “Why Invest in Universities” to add to the debate, and to reinforce the importance of universities not just to the economy, but more broadly.

As I often write, it’s important that we create a narrative about higher education, not just about the economic benefits to the individual , but just as importantly, the benefits which do not just accrue to the individual but also to wider society and the benefits that are not purely economic.

UUK suggests that the UK should invest in universities because:

• Universities transform people’s lives through education and through the wider impact of their research.
• Universities help students to develop the skills and knowledge employers need.
• UK university research is academically world leading and more cost effective than anywhere else in the world, providing the ideas and inventions on which future prosperity will be founded.
• University research benefits everyone – creating businesses and jobs, enriching society and stimulating culture.
• Universities help to ensure that the UK remains competitive in the global market by supporting greater business innovation and export-led, knowledge intensive growth.
• Universities’ international success helps secure the UK’s share of global growth and influence.
• Universities are anchor institutions in their regions – they are essential for vibrant local economies and are drivers of innovation and business development.
• Universities are major contributors to the UK economy, generating £73 billion of output in 2011 alone.
• Universities have transformed themselves in many ways over the past decade, including becoming more efficient and cost effective

Evidence is provided for each of these assertions, and it’s good to see yet again the commitment to showing the importance of international students to university income. The report suggests that 12% of university income nationally comes from international students, and identifies how this is importance in maintaining provision of certain STEM subjects. However, the report also acknowledges the growth in the market for international students in Australia, Canada and USA, who are expanding relative to the UK. Referring back to last week’s blog piece on the recent PA Consulting report on how well the UK sector can face global challenges, this seems to be one where external factors rather than internal issues are holding us back.

As we sit and wait for the changes to the BIS budget, this report helps identify the successes of UK higher education, identifying that universities want to:

1. Provide high quality education that meets the UK’s knowledge and skills needs
2. Provide opportunities for all people with the ability and motivation to study at university to be able to do so
3. Deliver world-class research and support innovation
4. Support the UK’s regions and provide opportunities for businesses to enhance their innovation capacity
5. Attract investment and talent from abroad, and maintain the UK’s international competitiveness in foreign markets

and specifically looking at the second of these, the report proposes that government should

“continue to invest, along with universities, in funding
to support social mobility; the allocation of this funding
should recognise the importance of attracting students
to university as well as supporting a diverse student
body while they are studying”

I think we are all waiting to see whetherr the funding for social mobility through the Student Opportunity Fund will continue – if it is reduced, this could have significant impact on those institutions who recruit large numbers of widening participation students, both in terms of their budgets, but more importantly in how they are able to continue to provide transformation learning.