After a quiet time on the wonk front, last week saw the publication of the White Paper and two new reports on employability of STEM graduates, announcement of a Higher education bill in the Queen’s speech, and the launch of the technical consultation on the Teaching Excellence Framework, not forgetting the previous week’s plans for consulting on the future of DLHE. Anyone would think that HE wonks had been twiddling their thumbs for a while, with nothing to critique or criticise. For a really good set of resources on this, it is worth looking at WonkHE.

The White Paper contained few real surprises – changes to quality arrangements, making it easier for new entrants to the market, the introduction of a teaching excellence framework, changes to the landscape and research support – all were previously consulted on in the previous Green Paper. Overall, the sector has not been unreservedly supportive, but even with a small parliamentary majority, the bill is likely to become law, and so we need to learn how we can work as well as possible within this revised landscape.

Overall, the changes are to drive further the marketisation of higher education – no matter how we might suggest that HE does not operate as a fully open market, the government is wedded to the idea that increasing competition will drive up quality. Hence, the idea that new entrants  – “challenger” institutions will be able to provide competition to existing incumbents. Similarly, the teaching excellence framework is touted as providing more information to prospective students, hence helping them to make more informed decisions. There is, of course, little evidence that students make decisions purely on data, and for many students, there may not be a free choice of where they study, based on financial circumstances, and family or work commitments.

Nonetheless, we will have a TEF, and so it’s important to understand what will drive success in this, so that we can get the best possible outcome which reflects our performance. One piece of good news is that the government did listen to the sector in terms of timing of implementation, even if concerns about the metrics to be used fell upon stony ground.

From the technical consultation, we know that the following principles should underpin TEF:

• keep bureaucracy and burden to a minimum
• be voluntary, allowing providers to consider the benefits and costs of applying before deciding whether or not they wish to
• allow for diverse forms of excellence to be identified and recognised
• support rather than constrain creativity and innovation
• respect institutional autonomy
• be based on peer assessment
• be robust and transparent
• result in clear judgements about excellence for students, employers and other stakeholders
• avoid driving perverse or unintended behaviours in the pursuit of demonstrating excellence
• be sufficiently flexible to allow for further development as the TEF evolves.

From year 2 of TEF, institutions who choose to be assessed can be judged to meet one of three outcomes: Meets Expectations, Excellent or Outstanding. To get to this, we would be assessed on: teaching quality, learning environment, student outcomes and learning gain.

And the part we need to be mindful of is how this will be assessed.

Teaching quality will be based on questions 1- 9 of the National Student Survey (teaching and assessment and feedback). Learning environment will be judged on questions 10-12 of the NSS (academic support) and non-continuation data from HESA, while outcomes will be assessed by the results of DLHE.

This does look remarkably like a league table, and so institutions will work harder than ever to make sure that their NSS results and DLHE figures show outcomes in the best possible light.

In addition to the data, providers will provide a written submission of no more than 15 pages. This is where we will be able to provide more context to what we do – examples cited in the document discuss: use of student surveys, collecting and responding to module feedback, staff development activities, timeliness of feedback, use of employers on validation panels, levels of contact time and independent study.

This is going to be a lot to cover in 15 pages, so it will be key for institutions to have their policies really clearly defined in terms of how their various mechanisms work, and how they can be shown to improve student experience and outcomes.

Our recent work on changing module evaluation processes and observation of teaching, and our review of quality processes will put us in a good position to explain how we manage our academic delivery to provide the best experience for students. We will clearly need to focus more on some of our student survey scores, and get to the bottom of why we have such a wide variety of reported experiences.

Next steps for us will be:  how we review our student survey outcomes; how we deliver our new employability strategy; how we ensure that we use the information from module evaluations and teaching observations to optimise student success, and how we review the performances of all of our courses.

There will no doubt be an ongoing resistance to TEF – the metrics chosen are still not ideal, and when we move to looking at subject level analysis, then there will be concerns regarding reliability of data – but this is a system we are going to have to work with. It would make sense to make sure we are best prepared as we can be.

The first of the major University league tables, the Complete University Guide, is published today.

This table uses metrics  on ten measures: Student Satisfaction, Research Quality, Research Intensity, Entry Standards, Student: Staff Ratio; Spending on Academic Services; Spending on Student Facilities; Good Honours degrees achieved; Graduate Prospects and Completion.

From the CUG press release:

Dr Bernard Kingston, principal author of TheCompleteUniversityGuide.co.uk, said: “There is a considerable degree of stability at the upper end of the league table this year. While dramatic changes may be newsworthy, this stability indicates that the rankings are robust and credible for young people seeking a university place – our primary purpose.”
This year’s release sees TheCompleteUniversityGuide.co.uk publish a number of new rankings. Alongside new subject tables for Creative Writing, Forensic Science and Occupational Therapy, there is now a Creative and Performance Arts table, containing 14 institutions that do not feature in the Main Table.
Dr Kingston said: “We have simultaneously released a survey of universities’ relative success in resolving student complaints. This shows significant variations between universities and is an important source of information for prospective students who what to know that their complaints will be effectively resolved.” (See attached Press Release and Table).
“Last year’s Higher Education Green Paper, Higher education: teaching excellence, social mobility and student choice, stated that applicants need access to robust, timely and objective information, based on criteria that are straightforward and easily understood.

So the top ten are:

Not really any surprises there. Staffordshire falls 6 places to 109th.

What is always of more interest are the big movers, both up and down, and the identification through reading the individual subject tables to see why these changes have happened.

So this year’s big winners are:

• Manchester Met – up 16
• Harper Adams – up 14
• Buckingham – up 14
• Liverpool Hope – up 14
• Sunderland – up 14
• Falmouth – up 12
• Winchester – up 12
• Edge Hill – up 11
• Middlesex – up 11

At the other end we have

• Oxford Brookes – down 11
• St Mark and St John – down 12
• Brighton – down 14
• Queen Margaret – down 15
• Royal Agricultural University – down 17
• Arts University Bournemouth – down 19

The section on complaints and their resolution will be of interest to academic registrars. Over a 3 year period, the number of completion of procedure letters issued after exhausting internal complaints process per 1000 students is ranked. The ranking here does not follow any meaningful pattern – it might be assumed that students at one type of university are more likely to use a complaints procedure than others, or that part of the sector would be better at dealing with complaints but this is clearly not the case. Pleasingly, the figure quoted for Staffordshire is considerably better than for some. Whether this data is of any meaningful us to prospective students is debatable.

As revealed in legislation last year, the government has been keen to see the impact of subject studied, and where, on the earnings of graduates.. The initial research has now been carried out by the Institute for Fiscal Studies, and looks at data that is more long term that the current DLHE data, and crucially considers student loan repayments and  tax returns.

Graduates from richer family backgrounds earn significantly more after graduation than their poorer counterparts, even after completing the same degrees from the same universities. This is one of many findings in new research published today which looks at the link between earnings and students’ background, degree subject and university attended

Having carried out the research, some of the the findings could be considered as underwhelming:

• students from wealthy backgrounds out-earn others, when studying the same subject at the same institution
• graduates in creative arts earn less than others.

Inevitably the reaction from some places has been to roll out the “more means worse” arguments, for instance here in the Daily Telegraph, where Fraser Nelson writes:

If a book is ever written about the mis-selling of higher education, it might start with such adverts. There’s no doubt that doctors and lawyers earn a bomb; no doubt that an Oxbridge degree opens many gilded doors. But studying urban dance at Peckham University or media studies at the University of Scunthorpe is another story entirely.

Yes, the average graduate premium may be generous. But today, all too many ropey institutions hide behind the word “university” – offering dismal courses that serve neither students nor society. And by the time the students realise that they’ve been sold a pup, it’s too late.

A more detailed reading of the paper would reveal that although there may be 23 institutions where the median salary for male graduates is lower than for non graduates (as shown in this almost indecipherable graph), the authors state:

At the other end of the spectrum, there were some institutions (23 for men and 9 for women) where the median graduate earnings were less than those of the median non  graduate ten years on. It is important to put this in some context though. Many English higher education institutions draw a signicant majority of their students from people living in their own region. Given regional differences in average wages, some very locally focused institutions may struggle to produce graduates whose wages outpace England-wide earnings, which include those living in London etc. To illustrate regional differences, employment rates in the period under consideration varied between66% in the North East and 75% in the East of England, and data from the Annual Survey of Hoursand Earnings suggests that average full-time earnings for males were approximately 48% higher
in London than in Northern Ireland, and around 34% higher for females. Regional differences are therefore important and we take them into account in our analysis of graduates’ earnings.

More interestingly though is how this data might be used in the future. In  this paper, the authors have not published results against all named institutions, although most of the Russell Group universities are named. In future, the intention would be to show this. One argument could be to use the data to allow differential fees, or to have differential RAB charges by subject or institution. Alternatively the information could be used to provide better student information and to challenge policies on social mobility. A recent article in the Times Higher looks at the different views from across the sector.

A clear message for us however might be to continue with our focus on developing students’ employability skills and being prepared to make sure that these skills which might currently be missing, are deeply embedded into courses or into extra-curricular activities. For instance, we can do more to develop numeracy and digital capability skills, by understanding exactly what it is that potential employers want to see in the graduates that they employ.

More challenging is around the issue of social capital. As a university that has at its heart a belief in education as a transformational activity, and a commitment to widening participation, we might do well to understand more how we can help our students develop social and cultural capital – without this they will always find t more difficult than those for whom university was an expected rite of passage. It’s very likely that for many students- especially those who are local or who commute in daily – that their sense of bonding capital is high. The corollary is that the level of bridging capital – that which they need to develop new networks – is lower than for students with different backgrounds. Identifying activities that will help our students develop this could be key. Some possible areas are placements, internships, and cross-disciplinary projects, where students have to work on real world problems but with student from other subjects, to pull them out of their comfort zone.

Over the next few weeks it will be instructive to see how politicians react to this new data, and from this for us to identify specifically what we should do to respond.

Last week HEFCE published their revised operating model for quality assessment. This is based on the responses from the sector consultation that took place last year, and where we, and many other universities, identified areas that were of concern to us. Some of these have been addressed. However, this is also part of the current sectoral land grab to have the responsibility for qualit; at the same time as publishing, HEFCE has put out to tender various aspects of its quality work.

Key points to note from the revised operating model:

• “future quality assessment arrangements should seek to encourage innovation in learning and teaching, rather than driving providers towards risk-averse activities and homogenised provision.”
• “approach for implementation is therefore designed to be proportionate, risk-based and grounded in the context of each individual provider and its students”
• a set of baseline regulatory requirements will still based on parts of the existing quality code and the framework fr higher education qualifications
• fore new entrants there will be a gateway process followed by a developmental period of enhanced scrutiny and support
• for established providers, a review of their own review processes, followed by a data-based Annual Provider Review and a revised periodic review visit

Some common areas of contention from responses from the sector were: comparability of standards; a potential national register of external examiners, and the roe of governing bodies.

A large section of the document covers comparability of standards, and classification algorithms used.The document states that when reviewing the original proposals:

Arguments mobilised against the proposals included:
• an opposition in principle to the funding bodies acting in an area where institutional autonomy is prized
• a view that there was no particular problem to be resolved, or that the specific proposals would not resolve whatever problems might exist
• a series of more practical concerns relating to increasing the burden on external examiners, thereby disincentivising the people on whom the successful operation of the system depends.

But that “student and PSRB respondents were much clearer that modernisation in this area was important, with some suggesting that the proposed reforms did not go far enough”

HEFCE have moved away from the proposal for a national register of external examiners, but talk instead of training of examiners to ensure that they are able to check comparability of standards – there is still a worry that good degree rates are rising and that these may not be defensible

The role of governors was an area that may universities had plenty to say about in the response to consultation, where it was felt that governing bodies may not be best placed to make direct judgements about academic quality. Again, HEFCE have clarified their expectation:

The role of the governing body would be to receive reports and challenge assurances from within the institution. It should not be drawn into quality management activities itself. We recognise the predominant role of senates and academic boards (or equivalent) in academic governance, and the responsibility of the accountable officer and senior executive team, and would expect an individual governing body to be clear about the formal relationships between the elements of the governance arrangements in its own institutional context.

There’s plenty more to digest. As always, WonkHe have a guide to how the new system will work, written by Louisa Darian.

What will be interesting now is the transitional arrangements and the pilots to be run during 2016-17.

In a piece on Vox.com, titled “How Amazon could destroy college as we know it”, Alexander Holt speculates on how Amazon could move into the business of HE. It’s based on an imagined speech by Jeff Bezos in 2030, reflecting on what Amazon has achieved. It’s based on imagination, but supported by references to actual achievements by Amazon. As with all service provision that has previously been primarily state or public funded, we know full well that venture capitalists and technology entrepreneurs see that HE is ripe for “disruption” and that technology will play a key part in this solutionism.

Holt envisions Amazon developing their already developed classroom used to support their staff, together with “badging” of competences to create mastery of tracks such as logistics. The next step proposed is that of the Amazon University to support internal staff development. So far, so similar to plenty of other in-house training or development schemes. The next imagined step is the interesting one – what if Amazon opened up these classroom, badges and courses to anyone? What if they offered access to Prime customers at no extra cost, to be able to study for a qualification and to keep them locked into the Amazon customer experience?

At a time when the MOOC-boosters have gone a little quiet (remember the heady days of 2012?), then maybe looking at those companies such as Amazon, or Google, (even Facebook) with their “closed garden” view of the internet, and their sheer dominance over the provision of web services, and maybe we can see the latest potential disruptor of higher education.

A couple of years ago I took a few MOOCs, one of which was on disruptive technologies. As a final assessment, I wrote a paper (essay in UK terminology) on how technology would disrupt HE. At a time when we are looking at the possible outcomes of a Green Paper review that was fixated on ideas of teaching excellence, but focusing entirely on metrics of students who have been full time undergraduates, then maybe we need to look again at how technology might (or might not) change higher education. At the time I argued that MOOCs wouldn’t be the game-changer that was being suggested at the time. However, we need to find a better use of technology that will be the key in helping to change HE, provided that it is used in a way to reduce gaps in inequality of access to learning, not to increase them, and  to enhance the learning experience of students in meaningful way. A blog post from 2014 revisited these ideas.

If you want to read the original essay, then a copy of it can be supplied!

An easy way for media outlets to fill space in the dog days running up to Christmas is to provide a review of the year. This blog is not going to shirk from that less than onerous task, as we look at what was reported, debunked, or analysed in these pages over the last year.

January 2015

The year started off with a look at the first league table out of the blocks – the People and Planet League Table. A bit of  a slide for us in this one, but as the Guardian reported at the time:

A number of universities seem to have become frustrated over time with the “green league”, which has also this year been renamed to remove the word “green” from the title. Concerns centred on the time involved in collating the information required, some criticisms of aspects of People & Planet’s methodology, and perceived goal-post changing

Also in January we looked at the UCAS data release for the previous year, which contained the surprising information that some universities have increased their number of applications, and that there is a gender divide between subjects.

February 2015

In February, we looked at a report which showed what MPs thought about universities – 3 months before an election, it seemed like a good idea:

“When asked about how well universities perform, then while 78% though universities did well at world leading research and 71% though they did well at competing internationally with other HE sectors, only 56% thought universities did well at producing highly skilled and employable graduates and 48% thought they did well at contributing to local employment and the local economy in their areas. More worryingly only 38% thought universities did well at using their funding efficiently (funding from their assets, students, the government and others)”

Also we looked at the numberr of good degrees begin awarded across the sector, new writing on BME success from the Runnymede Trust, the need to be CMA compliant and a report from HEPI, which led to my first quotation in the Times Higher and the following ideas:

• the increasing focus on employability – are we keeping pace with others in the sector on this?
• the development of graduate attributes – how distinctive are these between individual universities?
• the increase in use of  performance management tools – how do we ensure we have the right data, and use it for enhancement?
• provision of foundation year programmes – is the CUC model one that others might choose to replicate?

March 2015

Not much happened on the blog in March, apart from an article “Let’s Talk About Race”.

It’s something we still need to be talking about.

April 2015

In April, we reported that StaffsUni had improved in the Times Higher Student Experience Survey 2015 and had risen  rises 2 further places in the Complete University Guide.

Most prominent this month though, was the steady march towards teh General Election, and this included   a review of the major parties’ manifestos. Somewhat presciently, the Lib Dems were considered in this article under “The Others” – a rare bit of foresight into their likely election performance.

May 2015

May brought us a General Election, so in advance of this I produced a reflective piece ion what universities are for, and post-election wrote a piece on the changes we were likely to see. Pleasingly, this was republished by the Guardian, so luckily there was nothing too controversial.

Late in the month, the Guardian University League Table came out, with another rise for StaffsUni. This was the most read of all articles through the year.

June 2015

Starting with an article referencing Supertramp (song titles do appear frequently if you want to go searching), this month we looked at the annual PA survey of Vice Chancellors, who felt that the UK is lagging behind in every major area of innovation, and propose the following as the reasons for this:

1. deep seated conservatism of university cultures
2. constraints of inflexible organisational structures
3. fragmented and tentative nature of change initiatives
4. perceived lack of incentives for innovation
5. improved confidence in resilience of sector
6. widely held views that current models of HE provision and participation will remain the same for years to come

We also looked at the use of data – both in terms of the end to produce graduates who are numerate and data literate, but also to have university staff who can use data effectively.

July 2015

Graduation month for us here at StaffsUni, and another popular post for the year – a guide to staff on how to behave at graduation ceremonies, with such tips as:

• “You may have heard the same speech several times for the last however many years. Remember to laugh at the joke. Not too heartily”

and

• “If you can gatecrash the senior staff reception, then this is the place for the best snacks”

On a more serious note, we saw Jo Jhnson’s first major speech, as well as HEFCE launching its consultation into QA arrangements. HEFCE may have been premature, as Johnson announced the TEF, and hinted that QAA could be the ones to run it…..

August 2015

A quieter time of year, so another chance to look at the importance of numbers, and a review of The Metric Tide. This would come in handy later in the year when we saw the consultation on TEF, but more locally, I suggested that we should be getting good with data:

• To make sure all colleagues are aware of how measurable outcomes affect us reputationally and reflect the results and experience of actual students
• To provide a consistent reliable management information to act as a trigger
• To raise the data understanding capability of all groups of staff.

September 2015

A new academic year, and in a speech to UUK, Jo Johnson said “there is lamentable teaching that must be driven out of our system”. Based on no evidence whatsoever. However, this set out what we were about to learn in the Green Paper. My conclusions were:

• A commitment to great teaching won’t be argued with – the mechanisms of assessing it will be.
• The change in regulation for alternative providers might be seen as a threat to some institutions (probably only those in the bottom quartile of league tables, or current FE providers of HE)
• The focus on widening participation should be welcomed – provided that funding and full data analysis is part of the deal.

Also in September we saw a rise in our position in the Good University Guide, and in final piece on good degrees, I wrote that:

“As we move into a potential quality regime that could be metrics based, together with a Teaching Excellence Framework, which will certainly use a variety of metrics (possibly including learning gain), then there will be plenty of work to be done in generating data and analysing it..

However, the focus also has to go beyond analysing data. How can we use it to understand our students both as individuals and as cohorts? How can we use data to support our staff better in teaching and assessing their students? Finally, how can we learn to change practices and behaviours based on evidence?”

October 2015

This month, we looked at the politics behind TEF, and suggested that: “one of the unintended consequences that TEF might bring about is a gaming of the system. I’m not suggesting that data returns that feed into league tables are inaccurate, but one part of a successful league table result is a set of carefully constructed data returns. It’s equally likely that it will be possible to do something similar with any TEF submission, so all institutions will learn very quickly how to report data in the best possible way”.

The month ended with a detailed piece explaining the rationale behind our revised Learning and Teaching Strategy, that went out for final consultation.

November 2015

The month started with “Who are You?” – questioning who our students are, what they want, how well we know them, and how well we understand the reasons behind a rise in consumerist behaviour.

In the second week, we got the big story of the year, and every wonk blog started churning out pieces on the Green Paper, in particular, on the Teaching Excellence Framework. This blog, never one to miss a trend, was no exception.

This was followed by a piece on student satisfaction, with another song title to start, which suggested that we needed to:

• carry on listening to students, responding and being seen to respond to surveys
• make sure we focus on all the measures that make up a league table
• make sure that courses are well organised and running smoothly
• don’t expect league table moves to immediately be reflected in increased applications
• and remember – the student experience is what really matters, not the survey itself.

The research reported in this formed part of a talk given to our Academic Group Leaders that month, where we looked at a range of ways data could be used.

December 2015

The last month of the year saw a review of the most recent Equality Challenge Unit data. Still we see a gap in degree attainment for students who don’t come from a white background.

UUK published 2 major documents – firstly a look at trends in HE, showing an anticipated need for more people in the economy with master level qualifications, and a second piece on supply and demand for higher level skills, which provided useful business insight into the gaps between what universities are providing, vs what employers think that they want.

Summary

It’s been an interesting year in HE. The dominant narrative that a degree is primarily about enhancing employment outcomes (not employability) is being increasingly reinforced. The ideas around TEF mean a potentially bureaucratic behemoth will be created, which clever institutions will learn to turn to their advantage. Students increasingly behave as consumers, but within the sector we don’t always understand how we have contributed to this set of behaviours. Data, and using it well, is becoming increasingly important.

My blog stats showed that I’ve had over 11,000 hits on the site now, so I think I’ll carry on.

For the next year, I expect this blog will be covering:

• changes post-Green Paper consultation
• the need to use metrics appropriately
• the use of technology in learning and teaching
• league tables (again)

And of course, the use of 70’s song titles and references in articles.

Now as we look forward to the next years, this writer will leave the last word to Morrissey – Happy Christmas, everyone.

(from This Charming Charlie)