HESA Data Release

Posted on by

HESA have just published their statistical first release for student enrolments and qualifications obtained at Higher Education providers in the United Kingdom 2014/15.

This is always a useful summary, to see the size of the HE “market”, and whch subjects appear to be growing or in decline, data which of course can be cross-referenced to UCAS data releases to to see how trends in applications map to trends in enrolments.

The headline data shows nothing new – the total number of students engaged in HE study dropped by 2%, largely due to the 6% drop on part time enrolments. Part time still continues to be a problematic area for the sector.

hesa1415_chart_1

In terms of subjects, we can see how individual subject areas are growing or in decline, which should influence the way in which institutions might want to proactively manage their portfolio.

The latest information shows that the areas of growth for undergraduate study are: biological science, computer science, subjects related to agriculture, engineering and technology, with the biggest gain in creative arts and design. On the other hand, there has been a sector wide drop in enrolments at undergraduate level again in languages, but also in business, law, history and philosophy, and education.

hesa1415_chart_5

On attainment, and an area of interest in light of comments on possible grade inflation in the recent discussions around the Green Paper, HESA note that “of those gaining a classified first degree, the proportion who obtained a first or upper second has shown a steady increase from 64% in 2010/11 to 72% in 2014/15. In 2014/15, 22% gained a first class degree compared to 15% in 2010/11.”. This steady rise will be reflected in league tables of course, but importantly for my own institution, our good degree rate has risen (not to the sector average), but to a defensible level.

Looking at data n where students come from, we can see that the UK is still a desirable location for HE study. Considering English HEIs only, the data shows:

hesa14-15 domicile

Not surprisingly we see that China remains the biggest provider of students to English HEIs, and continuing drop in students from India, Pakistan and Saudi Arabia, while there has been a big rise in students from Hong Kong.

As always the HESA data release provides excellent background information for anyone wanting an understanding of the shape of the UK HE sector, and where the trends are in types of students, their level and mode of study, their domicile, their outcomes and the attractiveness of the various subject groups.

 

 

Review of the year – in blogs

Posted on by

An easy way for media outlets to fill space in the dog days running up to Christmas is to provide a review of the year. This blog is not going to shirk from that less than onerous task, as we look at what was reported, debunked, or analysed in these pages over the last year.

January 2015

The year started off with a look at the first league table out of the blocks – the People and Planet League Table. A bit of  a slide for us in this one, but as the Guardian reported at the time:

A number of universities seem to have become frustrated over time with the “green league”, which has also this year been renamed to remove the word “green” from the title. Concerns centred on the time involved in collating the information required, some criticisms of aspects of People & Planet’s methodology, and perceived goal-post changing

Also in January we looked at the UCAS data release for the previous year, which contained the surprising information that some universities have increased their number of applications, and that there is a gender divide between subjects.

February 2015

In February, we looked at a report which showed what MPs thought about universities – 3 months before an election, it seemed like a good idea:

“When asked about how well universities perform, then while 78% though universities did well at world leading research and 71% though they did well at competing internationally with other HE sectors, only 56% thought universities did well at producing highly skilled and employable graduates and 48% thought they did well at contributing to local employment and the local economy in their areas. More worryingly only 38% thought universities did well at using their funding efficiently (funding from their assets, students, the government and others)”

Also we looked at the numberr of good degrees begin awarded across the sector, new writing on BME success from the Runnymede Trust, the need to be CMA compliant and a report from HEPI, which led to my first quotation in the Times Higher and the following ideas:

  • the increasing focus on employability – are we keeping pace with others in the sector on this?
  • the development of graduate attributes – how distinctive are these between individual universities?
  • the increase in use of  performance management tools – how do we ensure we have the right data, and use it for enhancement?
  • provision of foundation year programmes – is the CUC model one that others might choose to replicate?

March 2015

Not much happened on the blog in March, apart from an article “Let’s Talk About Race”.

It’s something we still need to be talking about.

April 2015

In April, we reported that StaffsUni had improved in the Times Higher Student Experience Survey 2015 and had risen  rises 2 further places in the Complete University Guide.

Most prominent this month though, was the steady march towards teh General Election, and this included   a review of the major parties’ manifestos. Somewhat presciently, the Lib Dems were considered in this article under “The Others” – a rare bit of foresight into their likely election performance.

May 2015

May brought us a General Election, so in advance of this I produced a reflective piece ion what universities are for, and post-election wrote a piece on the changes we were likely to see. Pleasingly, this was republished by the Guardian, so luckily there was nothing too controversial.

Late in the month, the Guardian University League Table came out, with another rise for StaffsUni. This was the most read of all articles through the year.

June 2015

Starting with an article referencing Supertramp (song titles do appear frequently if you want to go searching), this month we looked at the annual PA survey of Vice Chancellors, who felt that the UK is lagging behind in every major area of innovation, and propose the following as the reasons for this:

  1. deep seated conservatism of university cultures
  2. constraints of inflexible organisational structures
  3. fragmented and tentative nature of change initiatives
  4. perceived lack of incentives for innovation
  5. improved confidence in resilience of sector
  6. widely held views that current models of HE provision and participation will remain the same for years to come

We also looked at the use of data – both in terms of the end to produce graduates who are numerate and data literate, but also to have university staff who can use data effectively.

July 2015

Graduation month for us here at StaffsUni, and another popular post for the year – a guide to staff on how to behave at graduation ceremonies, with such tips as:

  • “You may have heard the same speech several times for the last however many years. Remember to laugh at the joke. Not too heartily”

and

  • “If you can gatecrash the senior staff reception, then this is the place for the best snacks”

On a more serious note, we saw Jo Jhnson’s first major speech, as well as HEFCE launching its consultation into QA arrangements. HEFCE may have been premature, as Johnson announced the TEF, and hinted that QAA could be the ones to run it…..

August 2015

A quieter time of year, so another chance to look at the importance of numbers, and a review of The Metric Tide. This would come in handy later in the year when we saw the consultation on TEF, but more locally, I suggested that we should be getting good with data:

  • To make sure all colleagues are aware of how measurable outcomes affect us reputationally and reflect the results and experience of actual students
  • To provide a consistent reliable management information to act as a trigger
  • To raise the data understanding capability of all groups of staff.

September 2015

A new academic year, and in a speech to UUK, Jo Johnson said “there is lamentable teaching that must be driven out of our system”. Based on no evidence whatsoever. However, this set out what we were about to learn in the Green Paper. My conclusions were:

  • A commitment to great teaching won’t be argued with – the mechanisms of assessing it will be.
  • The change in regulation for alternative providers might be seen as a threat to some institutions (probably only those in the bottom quartile of league tables, or current FE providers of HE)
  • The focus on widening participation should be welcomed – provided that funding and full data analysis is part of the deal.

Also in September we saw a rise in our position in the Good University Guide, and in final piece on good degrees, I wrote that:

“As we move into a potential quality regime that could be metrics based, together with a Teaching Excellence Framework, which will certainly use a variety of metrics (possibly including learning gain), then there will be plenty of work to be done in generating data and analysing it..

However, the focus also has to go beyond analysing data. How can we use it to understand our students both as individuals and as cohorts? How can we use data to support our staff better in teaching and assessing their students? Finally, how can we learn to change practices and behaviours based on evidence?”

October 2015

This month, we looked at the politics behind TEF, and suggested that: “one of the unintended consequences that TEF might bring about is a gaming of the system. I’m not suggesting that data returns that feed into league tables are inaccurate, but one part of a successful league table result is a set of carefully constructed data returns. It’s equally likely that it will be possible to do something similar with any TEF submission, so all institutions will learn very quickly how to report data in the best possible way”.

The month ended with a detailed piece explaining the rationale behind our revised Learning and Teaching Strategy, that went out for final consultation.

November 2015

The month started with “Who are You?” – questioning who our students are, what they want, how well we know them, and how well we understand the reasons behind a rise in consumerist behaviour.

In the second week, we got the big story of the year, and every wonk blog started churning out pieces on the Green Paper, in particular, on the Teaching Excellence Framework. This blog, never one to miss a trend, was no exception.

This was followed by a piece on student satisfaction, with another song title to start, which suggested that we needed to:

  • carry on listening to students, responding and being seen to respond to surveys
  • make sure we focus on all the measures that make up a league table
  • make sure that courses are well organised and running smoothly
  • don’t expect league table moves to immediately be reflected in increased applications
  • and remember – the student experience is what really matters, not the survey itself.

The research reported in this formed part of a talk given to our Academic Group Leaders that month, where we looked at a range of ways data could be used.

December 2015

The last month of the year saw a review of the most recent Equality Challenge Unit data. Still we see a gap in degree attainment for students who don’t come from a white background.

UUK published 2 major documents – firstly a look at trends in HE, showing an anticipated need for more people in the economy with master level qualifications, and a second piece on supply and demand for higher level skills, which provided useful business insight into the gaps between what universities are providing, vs what employers think that they want.

Summary

It’s been an interesting year in HE. The dominant narrative that a degree is primarily about enhancing employment outcomes (not employability) is being increasingly reinforced. The ideas around TEF mean a potentially bureaucratic behemoth will be created, which clever institutions will learn to turn to their advantage. Students increasingly behave as consumers, but within the sector we don’t always understand how we have contributed to this set of behaviours. Data, and using it well, is becoming increasingly important.

My blog stats showed that I’ve had over 11,000 hits on the site now, so I think I’ll carry on.

For the next year, I expect this blog will be covering:

  • changes post-Green Paper consultation
  • the need to use metrics appropriately
  • the use of technology in learning and teaching
  • league tables (again)

And of course, the use of 70’s song titles and references in articles.

Now as we look forward to the next years, this writer will leave the last word to Morrissey – Happy Christmas, everyone.

snoopy

(from This Charming Charlie)

 

 

 

 

 

 

 

 

 

 

 

 

Skills and employability

Posted on by

This post looks at a couple of recent publications, which ask how many graduates do we need, what skills do graduates need, and where might the gaps be? As we are gong though the process of refining our strategic plan and operating model, and asking ourselves what kind of a university we will be, it’s important we understand what kind of people our graduates will be

A couple of months ago the Chartered Institute for Personnel and Development suggested that  as many as 58.8% of UK graduates were in non-graduate jobs. This was based on a research process of self-reporting, and even if this figure seems overly high, then clearly a number of graduates are not matched well with the roles in which they find themselves, which is not good for the individual, or for the organisation.

CIPD suggested that the findings raised questions about the size of the HE sector in relation to our labour market needs. They conclude with;

Policy-makers need to scrutinise the range of courses offered by the HE sector and seriously consider the social and private returns to them. We conjecture that they will conclude that, in many cases, public funds could more usefully be deployed elsewhere in the education and training system. Our findings suggest that the presence of a large HE sector will not necessarily lead to the attainment of the knowledge economy so beloved by successive UK governments.

It is worth remembering that many of the social returns that come from engagement in higher education are indeed social, and affect wider society and are not purely economic. In a marketised HE system, then policy makers do not have jurisdiction over the range of courses that are provided – mainly because in the main public funding is no longer directed to the course, but to the individual in the form of a loan

CIPD  make good points about the need for wider opportunities for post 19 education, and also point out that in previous generations, many would have felt “under-employed” in their role.

Last week University UK produced  a new report “Supply and Demand for Higher Level Skills”  which provides a counterpoint to the CIPD position, and again asks if there are too many or too few graduates, and looks at the relationship between their subject choices and the future labour market as well as considering employability skills.

There are 6 key recommendations from the report, summarised as:

  1. There is currently an undersupply of graduates that will
    continue into the foreseeable future;…..There needs to be a better understanding of why certain graduates become mismatched, which skills could prevent this and where they can be best attained
  2. universities and employers need to talk about ‘employability skills……..Universities and employers should jointly develop a ‘skills translation’ exercise to help all parties understand how and where these ‘soft skill’ principles can be practically developed and applied.
  3. there will be demand for a greater number of workers with higher – but not necessarily degree-level – qualifications…….Young people should have the opportunity to develop higher-level skills through a system of integrated pathways between the two forms of provision, one that provides a theoretical underpinning to technical knowledge and offers the chance for upskilling in line with economic, operational and technological change
  4. the sector needs both a clearer and a more granular understanding of the size and content of provision across both further and higher
    education
  5. in spite of a strong supply of STEM  students, there are continued shortages of highly-qualified workers in technical industries. Identifying – and mending – this obstruction in the talent pipeline is crucial
  6. there should be a heightened focus on skills utilisation. What sort of management and business practices best utilise higher-level skills and how can similar practices be adopted by firms of diverse sizes and sectors?.

The UUK report summarises a number of surveys looking at skills that employers look for in graduates, and the extent to which graduates do, or do not, have these. While recognising the difficulty of identifying relationships between the various surveys, the authors note that “there is consistency in that most surveys point to graduates’ lack of work experience and some combination of ‘necessary’ or job specific skills.”

UUK suggest that universities have two things to tackle in this regard – making sure that students gain the necessary employability skills at the same time as gaining subject level knowledge which will provide skills in critical thinking, analysis and creative thinking.

Crucially UUK identify that when universities talk about employability skills, for instance teamwork, communication, they don’t necessarily understand this from the perspective of an employer. Hence the recommendation for a skills translation exercise, to make sure that all parties understand what is meant, and what is needed.

UUK also looked at how graduates of different disciplines self-reported  their levels of skills, which provided the following:

uuksupply1

There are some clear message heres, then, for different subject areas to consider in terms of where the focus is needed in developing non-subject skills.

Another report published this week by HEPI was “Employability – degrees of value“, written by Johnny Rich, in which the emphasis is on understanding employability, not just employment, and Rich argues for a new framework of employability embracing knowledge, skills and social capital. A novel part of this report, and one that is rarely mentioned is the importance that social capital plays in securing employment, and in being able to develop employability.

Similar in part to the conclusions of UUK, Rich proposes a generic set of skills, the level of which might vary by subject This same set of skills can be mapped by employment role, and so mismatches can be seen. A consistent framework for employability is proposed, designed to reduce the burden on academics, and which would allow students to personalise their course and their skills profile according to their needs and ambitions. This is not that dissimilar to our own Staffordshire Graduate Employability Programme in theory, if not in delivery.

In drawing these three strands together we can see that as  a university we need to be mindful of what our students want and need, especially in how we can make sure they are best prepared for when they leave us: not just to be “job-ready” but to have the embedded deep employability skills that they will need for their whole careers. Some of the sections of our new Learning and Teaching strategy already refer to reviewing and updating our Graduate Attribute statements and referring directly to social capital needs.

As we move into the next phase of our strategy delivery, then with our increased focus on employability we can be working now to ensure that our Staffordshire Graduate statements remain meaningful, not just for us but for students and potential employers, and that we have a common understanding of what we mean when we talk about transferable skills.

When we work with ministry panels or advisory boards, or use external expertise in curriculum development, we need to make sure that we are asking questions not just about subject and technical needs of employers, but also gaining that deeper shared understanding of what transferable skills are needed, and how we can help to develop them.

We need to make sure that graduates from all disciplines have the right mix of skills, to make sure that they are not mismatched to employer requirements, and finally, we must ensure that our graduates gain in social capital while they study with us, to make sure that they can compete with, and indeed be better than, everyone else.

 

Patterns and Trends in UK HE 2015

Posted on by

An annual report from Universities UK looks at trends in the UK HE sector based primarily on statistics from HESA, and looking over a 10 year period.

This is a period that has seen significant change, and when we look at the proposals in the recent Green Paper, there is little to suggest that the pace of change will be slowing.

A quick review of the report then gives us some business insight in to the sector generically, from which we can extrapolate for our own institution.

The three chapters consider: students, staff and finances. This blog piece will focus on students

Under students we can see the changes in make-up of the student population – full time numbers, both postgraduate and undergraduate, show an increase, while part time study clearly has declined.

uuk1

The contiued and drastic decline in PTUG is explained by

the removal in 2008–09 of funding for students taking qualifications equivalent to or lower than ones which they already had, and by reforms to undergraduate funding in 2012–13, including an increase in fees following cuts to teaching grants and issues around eligibility for tuition fee loans. At the same time the economic downturn has also caused a reduction in the number of students able to self-fund part-time study, and a reduction in the number of employers willing to support employees through part-time study.

The suggestion in eh recent Green Paper on removing ELQ restrictions for thise wishing to study STEM subejcts may help slightly in this area, but the contiued decline in part time numebrs alsoe reduces the chances of universties being able to tackle issues of social mobility, again a key part of the Green Paper.

While undergraduate numbers overall have increased, this increase is due to the numbers studying for a first degree. At the same time, the numbers enroling into other undergraduate courses has dropped significantly:

uuk2

Which would seem to have some pretty strong messages abut the worth of developing a number of courses!

On international student numbers, despite growth in the numbers of students who travel to study, there has bee a decline in the UK of international students, as we lose out to other markets such as USA and Australia.

uuk3

We may be seeing the impact of the UK view of students being part of the net migration target and the removal of PSW visas. As UUK highlight:

the decline in students from India are fuelling concern about the UK’s ability to attract international students, which could damage the higher education sector.
International student fee income accounted for 13% of sector income in 2013–14, and demand from international students can support the provision of certain strategically important subjects in the UK (eg engineering, technology and computer science, particularly at postgraduate level where around half of all students are from outside the EU). Higher education is important to the wider UK economy, with non-EU students contributing £7.2 billion to the UK economy and supporting around 137,000 jobs through their tuition fees, accommodation and off-campus expenditure in 2011–12.

Looking at subject of study over the 10 year period shows some clear trends, much of which is backed up by our own detailed analysis of UCAS data.

uuk4

The most surprising part of this is perhaps the overall drop in computer science, at the same time as students with god computer science skills are desperately needed by employers. The huge drop in subjects described as “combined” should not come as a shock – maybe students are looking for courses that they believe gofer a clear route to employability? Interesting, Keele, our nearest geographical competitor, is now developing courses in Liberal Arts and Natural Sciences, which initially might seem to fly in the face of this trend.

The report notes the number of people in the workforce who now have tertiary level skills, but recognises that the UK lags behind other countries in this regard. Also, looking to the future, a clear need is identified for an increased number of people with undergraduate and postgraduate skills for the needs of the international workforce of the future.

uuk5

This rise in demand for those with postgraduate qualifications ins a clear opportunity, and the recent Green Paper proposals for plans to make loans available to students wishing to enter PG study provides a clear opportunity.

The paper ends by summarising changes in student numbers, income and staffing:uuk6

and concludes:

Despite the challenges of a dynamic and sometimes uncertain operating environment, the UK higher education sector can point to a number of successes over the period……In this increasingly competitive international environment and with a referendum on remaining in the EU taking place before the end of 2017, the UK higher education sector will continue to work hard to maintain its position and attract the staff, students, funding and partnerships that are central to its success.

 

 

Equality Unit Statistical Report 2015

Posted on by

This year’s Statistical Report from the Equality Challenge Unit has just been published, and provides a wealth of detail on students and staff in the academic year 2013-14.

A main pat of my work has been looking at student attainment, and others in the University are working on addressing the attainment gap between white students, and those from BAME backgrounds.

Looking at the national data then, for 2013-14, we can see that:

“the ethnicity degree attainment gap in the UK was 15.2 percentage points. 75.6% of white qualifiers received a first/2:1 compared with 60.4% of BME qualifiers”.

and

“The ethnicity degree attainment gap has decreased from a peak of 18.8 percentage points in 2005/06 to 15.2 percentage points in 2013/14 – the lowest it has been in the last ten years.”

However, BME students cannot be considered as one continuous group, and variations exist between those of different ethnicities, as the graph below shows:

ecu151

The attainment gap exists for all groups, but has closed the most for Asian students. We can now benchmark our own degree outcomes for 2013-14 against this data, and also our 2014-15 outcomes to see how we are progressing compared with the sector overall.

The report also looks at the impact of subject on attainment and ethnicity and states: “In every subject, a higher proportion of white qualifiers received a first/2:1 than BME qualifiers.”

Considering students with a disability, then the attainment gap is nowhere near as pronounced as it with with students from a BME background.

ecu152

Finally the report considers the impact of  gender, and shows that female students were more likely than male students to gain a 1st or 2(i) and this was the case in all subject areas except social studies.

The report goes on to look at the intersections between the main groupings of data. One factor that is not considered is Widening Participation status. Although this is not a factor that is considered from an equality perspective, it is one that is raised as a possible factor whenever we talk about the attainment gap for BME students.

Ideally we need to be able to build models that allow us to visualise the outcomes of students, and to easily filter based on multiple characteristics. I’ve been doign abit of work on this very thing this week, and using some old data have built  Excel models that allow a visualisation of degree outcomes for combinations of ethnicity, WP status, disability and subject. It’s a great way of getting an insight into how different groups perform in different subject areas.

mghdegrees

Overall however, as a sector we still have a huge challenge to face in tackling the BME attainment gap. I’ve been writing about it for a number of years on this blog. I really look forward to the day I don’t need to write about this subject.

 

 

Autumn Statement 2015

Posted on by

As in previous years, the Chancellor’s Autumn Statement has provided a clear steer for the future of HE. For those who don’t see HE as being driven by the Treasury view, then look no further to the announcement in 2013 on the scrapping of Student Number Controls.

This year the headlines for HE are:

International students: numbers to grow;dependants of postgraduates on courses lasting more than a year will be welcome to come and work.

Widening participation: work with the Director of Fair Access to ensure universities take more responsibility for widening access, including collaborating on outreach to reduce inequality in admissions

Postgraduate support: lift the age cap on new loans to postgraduates from 2016-17 so they are available to all those under 60.

Part time support: introduce new part-time maintenance loans from 2018-19 to support the cost of living while studying.

ELQs: For all STEM subjects, tuition loans will be extended to students wishing to do a second degree from 2017-18

Widening the range of providers: £20 million competition to set up a new Institute of Coding; create a new university in Hereford focused on engineering in 2016; help to fund the £100 million development of a new campus in Battersea for the Royal College of Art.

Research: protecting today’s £4.7 billion science resource funding in real terms; long term science capital commitment of £6.9 billion between 2015-2021 to support the UK’s world-class research base; introduce a new body – Research UK – which will work across the seven Research Councils, review of the Research Excellence Framework

For detailed analysis of the figures, and in particular the implications for the changes to student loans , then read Andrew McGettigan over on Wonkhe.

So – area of concerns: two leap out straight away, firstly, there is the change to support for students in nursing and midwifery from the 2017 intake, who will now access loans in the same way as other student. This may have an impact on attractiveness of such courses, although in other areas the change to funding did not, overall, lead to a decline in applicants.

The second area of concern is around the student opportunity fund. The focus on social mobility is strongly articulated in the recent Green Paper, and the opportunity for funding for part time students and those wishing to study WLQs can be seen as part of the same narrative. A worry for a while has been that the Student Opportunity Fund within the  BIS HE budget was an easy target   The fund in itself is safe for now but:

The government will work with the Director of Fair Access to ensure universities take more responsibility for widening access and social mobility, and ask the Higher Education Funding Council for England to retarget and reduce the student opportunity fund, focusing funding on institutions with the most effective outcomes

And again, this could be seen as part of the continuum with the Green Paper, where some of the proposal for the higher levels of Teaching Excellence Framework will reward those institutions who perform well on this agenda.

Those universities that receive significant amounts of SOF will be looking hard at their income and the possible related outputs – here I’ve drawn together the 15-16 funding allocations, with data from league tables on entry tariff, good degrees, and degree completion. More detailed analysis will be needed to consider how to measure part time students, but a few hours playing with Heidi data might reveal how individual institutions are performing

wp allocations

As I’ve written before when looking at TEF, this is another example of a great opportunity for institutions to get clever about how they can use data to measure their performance, not just for the sake of creating a set of metrics, but as a way of measuring the efficacy of interventions, and using this to provide better evidence-based decision making.

There are some opportunities in this autumn statement for us as well – for example the more welcoming statement about  international students, although this still conflicts with the broader negative views around immigration, and for us would rely on assuming that students in our target market have dependents.

Availability of loans for postgraduate study is to be welcomed, increasingly higher level qualification will be needed for certain routes into employment, or as ongoing staff development.

The changes to availability of loans for part time study should also be welcomed, as indeed should the opportunity of funding for ELQs in STEM subjects.

Overall – it could have been a lot worse for HE – the BIS budget wasn’t cut as much as predicted, but with the Green Paper currently out for consultation, we probably have enough on our plates to be going on with!

 

 

Presentation to Academic Group Leaders

Posted on by

We regularly hold a forum at Staffordshire University for our Academic Group Leaders – these are the senior academic staff who are responsible for line managing and leading groups of academic colleagues.

This week I led the forum, with a presentation on league tables and on some of the implications of the recent Green Paper “Fulfilling our Potential. Teaching Excellence, Social Mobility and Student Choice“.

The slides can be viewed here at Slideshare

 

I can’t get no satisfaction

Posted on by

Early in January the next round of National Student Survey begins across the UK HE sector. This year for many will be seen as  a dry run for what is to come in later years – the widely discussed  Green Paper which refers to using metrics to help gauge teaching excellence. Once we get past the first year of everyone being equally excellent, then  NSS and DLHE are widely anticipated to be key measures to be used, as well as possible measures of learning gain, since the paper does hint at a lack of satisfaction with current degree classification processes.

So before we check on progress on last year’s action plans, and start to think about how we introduce this year’s survey to our students, a couple of publications from the last week are worth bearing in mind.

Firstly a research paper from QAA, written in part by Jo Williams of University of Kent, to which Staffordshire University made  a contribution. In this, “The Role of Student Satisfaction Data in Quality Assurance and Enhancement: How Providers Use Data to Improve the Student Experience” Dr Williams looks at how different types of institutions approach NSS, and shows that across all parts of the sector, institutions and senior staff questions NSS:

In particular, the issue of question 22 of the NSS, asking students about their overall satisfaction has created endless debates in academia, if not confusion, with professionals arguing that it is methodologically and professionally wrong to base league tables on a single question which is not in itself clear. Various senior academics we spoke to concurred with this theme.

The research did show that universities in all parts of the sector listen to what students say, and that they do make changes based on what the survey reveals, for instance:

The programmes change every year so sometimes it’s because the subject changes but very often it’s because students have expressed discontent with something. Therefore, you change the personnel teaching it. You change the way you teach it. You change the content, you change the assessment, you change the feedback, you change something about it. Or sometimes you just drop it.
(University B)

Changes in practice were noted across institutions:

Our data revealed that institutions have employed various changes in response to issues students raise in the satisfaction surveys. Among other practical changes, universities have:
– recruited academic advisors and officers to take charge of the NSS
– mapped internal surveys to mirror the NSS
– renewed their focus on learning and teaching revisited and improved timetabling systems
– raised structures including building sites, teaching rooms and sports complexes
– revisited their feedback and assessment mechanisms
– organised briefings with students to enlighten them about feedback and assessment, the NSS and its benefits
– replaced subjects, at times personnel, whose NSS scores keep falling
– introduced or empowered various forums and platforms to meet several times in a year to discuss the NSS, among others. Such forums found at nearly all institutions include: NSS forums, student experience action plans, education boards, NSS improvement forums and learning and teaching advisory groups

Across the institutions in the research, other similarities were see in how data was used: for instance comparing scores across schools, holding low scoring schools to account, and comparing with other institutions.

In terns of league tables, depending on where you appear in a league table appears to influence the behaviour of the organisation.

In particular, institutions placed in the top 25% of the league tables appear to have a relaxed view of the NSS. They appear to put particular emphasis on improving the student experience and argue that this automatically triggers a higher satisfaction rate than being ‘obsessed with the NSS’ and improving league table position:

Whereas at the other end of the scale:

In contrast, institutions in the lower 25% of the student satisfaction league tables appear to place particular focus on improving their student satisfaction and subsequently their standings in the league tables.

The main conclusion of the work then is that:

In particular, institutions in the top 25% of league tables
(Universities A and B) appear to prioritise improving the student experience and let the NSS take care of itself, while those in the bottom 25% (Universities C and D) prioritise their NSS league table position and subsequently employ various tactics to promote the surveys.
Despite institutions adopting different approaches to the surveys based on league table positions, institutions generally listen to students’ demands raised in surveys and have responded by instigating various changes including recruiting academic advisers and officers to take charge of the NSS; mapping internal surveys to mirror the NSS; raising structures including building sites and revising their feedback and assessment processes.

What the paper doesn’t consider is the relative ranking of NSS scores by institutions – it is perfectly possible to score well on certain NSS scores, and appear to out perform other institutions on such a single measure, but this may not change institutional behavours which may be set to focus on the NSS position, rather than overall experience.

In other work out recently, from Stephen Gibbons, Eric Neumayer and Richard Perkins writing in the Economics of Education Review “Student satisfaction,league tables and university applications: Evidence from Britain” (S. Gibbons et al./Economics of Education Review 48 (2015)148–164), the authors make the following points:

  • NSS scores have an impact on recruitment applications, but not huge
  • students do not appear to respond directly to quality cues from satisfaction scores
  • students may already have a well developed knowledge about product quality based on perceptions of reputation and prestige
  • student satisfaction and league table position do not have a short term effect on market demand
  • the degree to which quality indicators affect demand is strongly linked to the amount of market competition for a given subject

Finally, in “Applying Models to National Surveys of Undergraduate Science Students: What Affects Ratings of Satisfaction?”  (Educ. Sci. 2013, 3(2), 193-207) by Langan Dunleavey and Fielding of Manchester Metropolitan University, the authors look at what influences the results seen for question 22 – overall satisfaction. We are all familiar with reading through a set of results, with great scores for most of the questions, and sections, but a lower score for this final crucial question, which is the one used in all league tables.

The authors note the year on year consistency of results for individual subjects, noting how comparisons should be made:

Subjects were highly consistent year on year in terms of their relative performance in the satisfaction survey. This has implications for institutional decision-making particularly if subjects are wrongly compared against institutional averages, when comparisons should be made within subject areas (e.g., comparing with national subject averages, although this may be subject to error if courses contain different compositions of learners, for example in terms of ethnicity)

This is consistent with HEFCE advice, and why as an institution we provide sector average scores at JACS3 subject level for comparison.

Interestingly, questions about feedback were the weakest predictors of “Overall Satisfaction” whereas:

The best predictor of student satisfaction nationally for the three years analysed was “The course was well designed and running smoothly” followed by ratings of “Teaching”, “Organisation” and “Support”. This may vary slightly between subjects/institutions, so it is proposed that this type of quantitative approach to contextualising survey metrics can be used to guide institutions in resource allocation to tackle student experience challenges.

So our conclusions on how we approach the next NSS, and perhaps more  importantly NSS2017  could be:

  • carry on listening to students, responding and being seen to respond to surveys
  • make sure we focus on all the measures that make up a league table
  • make sure that courses are well organised and running smoothly
  • don’t expect league table moves to immediately be reflected in increased applications
  • and remember – the student experience is what really matters, not the survey itself.

Wonk’s Delight

Posted on by

And so the waiting is over. Friday finally saw the long awaited release of one the  most anticipated artifacts of 2016. Twitter was inundated with commentary, and major newspapers rushed to add reports to their online editions as the new John Lewis Christmas advert became available.

In other news, the government published its green paper – “Fulfilling out Potential – Teaching Excellence, Social Mobility and Student Choice”.

Much of the content of this was trailed previously in speeches by Jo Johnson, and last week on the Conservative Home website. Plenty of  excellent commentary has already been provided through WonkHE and I’d encourage my colleagues  to read these as well as the paper itself.

Over on WonkHE I’d recommend the following:

David Kernohan writes on the  first TEF’sassessment

Martin McQuillan also focuses on TEF, and considers that universities brought TEF upon themsleves

Andrew McGettigan crucially shows ten things you might miss on a first reading

And in the Times Higher, Jo Williams questions the impact that TEF will have on how universities look at student satisfaction, and how this could affect academic freedoms.

There are some key points for us all to know about the contents of the Green Paper, specifically around the emerging detail of the Teaching Excellence Framework, but also the longer term ambitions for HE in this country. I’ve previously suggested in this blog that the changes are intensely political – they are about creating a deregulatory regime (although that doesn’t quite tally with the regulation that TEF might create), with opening up the “market” further to new entrants, and to allow more rapid shifts in market share. This seems to be driven by a belief that teaching in universities is poor (with little evidence of this) and that institutions need to be able to show better value for money to consumers. The assumption clearly is that HE does work as a market – whereas many including me, would argue that it is not a traditional market, that students are not passive consumers, that they should be active participants in creating their learning, and that universities themselves have created this dumbed down “the customer is right” approach.

Notable in the document is an emphasis on full time undergraduate students Part time barely gets a mention, and although there are nods of acknowledgement occasionally that the sector is more complex than FTUG, this barely affects proposals

Let’s look at the key proposals, on TEF in particular, and what they will mean for us in the short and medium term, remembering that this paper is now open for consultation.

Part A covers teaching excellence, quality and social mobility, and this is probably the section of most interest to most university teaching staff.

Excellence is defined in the paper as:

There is no one broadly accepted definition of “teaching excellence”. In practice it has many interpretations and there are likely to be different ways of measuring it. The Government does not intend to stifle innovation in the sector or restrict institutions’ freedom to choose what is in the best interests of their students. But we do think there is a need to provide greater clarity about what we are looking for and how we intend to measure it in relation to the TEF. Our thinking has been informed by the following principles:
• excellence must incorporate and reflect the diversity of the sector, disciplines and missions – not all students will achieve their best within the same model of teaching;
• excellence is the sum of many factors – focussing on metrics gives an overview, but not the whole picture;
• perceptions of excellence vary between students, institutions and employers;
• excellence is not something achieved easily or without focus, time, challenge and change.

Ultimately there will be different levels of teaching excellence – once the metrics and process have been decided –  however, in the first year, any institution that has a currently satisfactory QA review will qualify for level 1 TEF.

In the short term then, we, along wth nearly everyone else, would “pass” the TEF.

Later there will be up to 4 levels of TEF, and institutions will apply for the various levels. This will be done either on a rolling process or when unis decide that they are ready to apply for a higher level. This is where is starts to be contentious and competitive

A key driver of TEF is to link fees to teaching quality, and so in future the fee cap will likely be set by government for each of the different levels created, so that fees can once again be seen as a differentiator – the current system has led to nearly all traditional providers charging the maximum of £9000, so this has not helped generate the anticipated market behaviours. Whether having 4 values of fees will have any more effect remains to be seen.

The paper proposes that the metrics to be used  initially to be: DLHE, NSS and HESA continuation and retention. You might as well read a league table!

However, importantly, the paper recognises the limitations of metrics alone:

“However, we recognise that these metrics are largely proxies rather than direct measures of quality and learning gain and there are issues around how robust they are. To balance this we propose that the TEF assessment will consider institutional evidence, setting out their evidence for their excellent teaching.”

The institutional evidence might look a lot like the self evaluation submissions for quality activities, In addition to this, there is an expectation of using further new common metrics on engagement with study (including teaching intensity) and learning gain.

Degree classifications and grade inflation cited as a concern, with a nudge towards a move to GPA, but without explicitly asking universities to do so. The numbers of good degrees award nationally has risen dramatically over the last 10 years – partly because of good teaching, partly through a different student attitude to study and the need for a higher classification, but maybe partly through gaming the algorithms to gain league table position

On social mobility, the attainment gap between BME and white students is highlighted, as well as the lack of engagement with disadvantaged white males. These are areas that would be expected to be addressed in the TEF – “Work to improve access and success should have close links with the TEF.”

Other key headlines to be aware are:

  • removal of barriers to entry for new providers, including a possible remval of teh minimum numbers of students.
  • creation of an Office for Students – as part of a bonfire of quangos the work of HEFCE and OFFA would move a new OfS. The OfS would be responsible for assessing quality of teaching through TEF
  • proposed  to constitutional arrangements of Higher Education Corporations which will affect us
  • reducing complexity of research funding, although this will be detailed more in the coming Nurse review

So what do we need to do?

Firstly, a lot of people (everyone who works in the university) should read the Green Paper. This is clearly setting out this government’s ambitions for the future of the sector.

Secondly, a robust response to the consultation is needed – where there are sector concerns, for instance over the detail of TEF, we need to echo and articulate those.

We also need to accept that this is coming our way, so we can start preparing in anticipation now.

Ideally, we should be creating systems and processes for capturing information both qualitative and quantitative, as part of our business operating model, so that we don’t have to do this as a separate later exercise, but build it into standard practice (at the same time as remving other less useful and time consuming processes).

Since we know the metrics to be used in the second year, then we can start benchmarking ourselves against other universities using the data sets in Heidi. We can also do the same with the Unistats data

We can start amending our own portfolio performance review tools to reflect the same information but at subject level. This could be considered gaming the system – but we all know that’s one of the contributory factors in winning in the league table game, and this will be no different.

We can set up course monitoring processes such that detailed data and analysis is readily available to support the work we do on supporting students from widening participation and BME backgrounds – we know we work well in this area, but will want quantitative as well as quantitative evidence

We can start to articulate what does make for excellent teaching so that we are ready to develop our institutional qualitative submission – maybe a review of our ongoing course approval and review process can be carried out to start to capture this now.

Overall, a lot of work needs to be done, firstly to understand the devil in the details of the green paper, to identify how to set up processes and practices to deliver the information needed. The agile organisation will be working out how to do this without creating any extra burden, by removing other activities that are no longer necessary, and making sure that academic staff have the space to focus on the most important work – teaching, and scholarship and research that enables them to be excellent teachers

Most importantly though, is the need for us to keep demonstrating that excellence is not easily measured; that education is not just a product for consumption; that higher education is the  opportunity for people to engage in transformational activity which will provide benefits to themselves as individuals, as well as to broader liberal society.

 

 

 

 

Who are you?

Posted on by

Not the chant at a football ground, but a question we could be asking of ourselves – how well do we know our students?

At an institutional level, or indeed course level we can do two  things. Firstly we could carry out market segmentation that might tell us so much – games designers like to play computer games in their free time for instance. Secondly we can realise benefits through a data-centric approach to supporting teaching. If we can identify for instance, that for a given module, more than 50% of the students taking it entered university with BTEC qualifications, then maybe there are some conclusions for the lecturer to draw in terns of how to deliver and in particular how to assess. As we move into the era of TEF, then this ability to use data to measure outcomes might be come ever more important, with the comments made recently by Jo Johnson on the differential performance of students from WP or BME backgrounds

In terms of having a holistic view of who we teach, are there some generalities we can consider to give a holistic view, and will this help us to understand some of the current trends in higher education?

The dominant narrative in HE is always about 18-21 year-olds who are studying full time. These are the students who, for most institutions, generate most of the income, and also provide the outcomes that are subsequently used in league tables and are most likely to be used in the Teaching Excellence Framework. While recognising the importance of other groups of students, this article focuses on this grouping, since their behaviours tend to dominate policy.

Firstly we can think of how this generation of students can be defined in terms of their attitudes to technology. Born in 1997, they have never used a computer that was not controlled with a mouse or a touch interface. They have been used to having a mobile device all their lives, and a tablet computer for the last 5. So through senior school and public exams, they have been used to having easy access to the web. They can’t even envisage a time when the web wasn’t ubiquitous. They operate in a  world defined by online social media networks and physical storage is a strange concept. Clearly for us to be able to work with these students, we need to have our own high levels of digital capability, and this is a topic I will be returning to in coming weeks.

As well as understanding the technology expectations that these students bring to university, we can also look at some of their motivations. Focusing on young full time undergraduates, then they are what was termed in today’s Observer as  Generation K, or Katniss. (You may need to have teenaged children to understand this reference.)

…the economist and academic Noreena Hertz, who coined the term Generation K (after Katniss) for those born between 1995 and 2002, says that this is a generation riddled with anxiety, distrustful of traditional institutions from government to marriage, and, “like their heroine Katniss Everdeen, [imbued with] a strong sense of what is right and fair”

This generation worry about getting a job and this is understandable after major economic downturn, but the impact it might be having on these students’ attitude to higher education education is that they perceive HE as a means to getting a job. Although this is understandable, within the academy we still have a responsibility to show that higher education is more than that.

Writing in the same newspaper, WiIll Hutton, author of “The State We’re In” and now Master of Hertford College, Oxford, tackles the issues of the freedom to argue and the freedom to be challenged within universities. Hutton is primarily concerned with the retreat of liberalism and the diminution in importance of the idea of a public realm:

This disdain for notions of publicness has created a vacuum occupied by the rise of a libertarian individualism that indulges belief over reason. Non-falsifiable belief systems used only to be the hallmark of ideological communism or religious zealotry. Now, ideas, especially on the right but also to a degree on the left are less and less tested in a public realm by debate, with evidence marshalled to justify them. Instead, they are asserted as valid because the holder believesthem.

This new individualism, alongside the decline of the public, has provoked a mounting tide of, at best, siloed thinking, impervious to criticism, and, at worst, the indulgence of rank prejudice. Thus, on the right, if I feel that Britain is being swamped by immigrants, climate change is bogus, maleness is being overwhelmed by “femininazi” women or the welfare system is transfixed by cheating, then, whatever the facts, my feeling is valid because I hold it. That suffices without proof or evidence. In any case, there will always be some article in a rightwing paper to justify it.

Again, this individualism can be expressed by students – being risk-averse and not willing to be challenged or taken out of their comfort zones leads to behaviours that create “safe spaces” on campus, where we can no longer  hear certain views in case others are offended. In the same way, we might have a group of students who don’t want to be challenged too much in their learning, seeing a degree as a commercial transaction, rather than a transformational experience, the benefits of which may only be realised many years after completion. We ask students every year in the National Student Survey whether they felt their course was intellectually stimulating. We want our students to engage with their discipline, not as passive recipients, but as active scholars providing contribution and challenge. To do this though, we will need to remind ourselves of the tendency to be risk-averse, to want to be treated as a consumer and to be challenged just enough – but not so much it becomes too hard. We would do well to remember that as well as being stimulating, courses are supposed to challenge, be intellectually difficult  and to provide arguments that question individual beliefs.

Finally, this generation of students perceive themselves to be consumers. As argued in her book “Consuming Higher Education”, Joanna Williams of University of Kent argues that this attitude to consumption doesn’t come about just because of the introduction of tuition fees. Instead, Williams argues that we have a generation who have been led into this interpretation of what higher education is about, in part by the behaviours of universities:

“Rather than universities challenging the idea that a degree is an entitlement, institutions instead strengthen this notion. The provision of quantifiable information on contact hours assessment patterns and employment prospects suggests students are correct to to perceive of a degree as a product.”

Individual institutions can’t turn around and say we are having no part of this agenda as we are obliged to publish this information and no doubt will provide more in future. What we can do is explain better what the quantifiable data actually means, and more importantly, what it does not mean.

As we all write our responses to consultations on quality assurance, on the teaching excellence framework, and as we redevelop our own strategy documents in the light of a forthcoming Green Paper, we need to remind ourselves that universities do exist as more than degree factories that are there to produce “satisfied” customers. Understanding more about the backgrounds of our students and recognising the reasons for their consumerist behaviours may help us in the long term to be able to better articulate what the wider benefits of HE are, and the role of universities in liberal society.