Presentation to Academic Group Leaders

We regularly hold a forum at Staffordshire University for our Academic Group Leaders – these are the senior academic staff who are responsible for line managing and leading groups of academic colleagues.

This week I led the forum, with a presentation on league tables and on some of the implications of the recent Green Paper “Fulfilling our Potential. Teaching Excellence, Social Mobility and Student Choice“.

The slides can be viewed here at Slideshare

 

I can’t get no satisfaction

Early in January the next round of National Student Survey begins across the UK HE sector. This year for many will be seen as  a dry run for what is to come in later years – the widely discussed  Green Paper which refers to using metrics to help gauge teaching excellence. Once we get past the first year of everyone being equally excellent, then  NSS and DLHE are widely anticipated to be key measures to be used, as well as possible measures of learning gain, since the paper does hint at a lack of satisfaction with current degree classification processes.

So before we check on progress on last year’s action plans, and start to think about how we introduce this year’s survey to our students, a couple of publications from the last week are worth bearing in mind.

Firstly a research paper from QAA, written in part by Jo Williams of University of Kent, to which Staffordshire University made  a contribution. In this, “The Role of Student Satisfaction Data in Quality Assurance and Enhancement: How Providers Use Data to Improve the Student Experience” Dr Williams looks at how different types of institutions approach NSS, and shows that across all parts of the sector, institutions and senior staff questions NSS:

In particular, the issue of question 22 of the NSS, asking students about their overall satisfaction has created endless debates in academia, if not confusion, with professionals arguing that it is methodologically and professionally wrong to base league tables on a single question which is not in itself clear. Various senior academics we spoke to concurred with this theme.

The research did show that universities in all parts of the sector listen to what students say, and that they do make changes based on what the survey reveals, for instance:

The programmes change every year so sometimes it’s because the subject changes but very often it’s because students have expressed discontent with something. Therefore, you change the personnel teaching it. You change the way you teach it. You change the content, you change the assessment, you change the feedback, you change something about it. Or sometimes you just drop it.
(University B)

Changes in practice were noted across institutions:

Our data revealed that institutions have employed various changes in response to issues students raise in the satisfaction surveys. Among other practical changes, universities have:
– recruited academic advisors and officers to take charge of the NSS
– mapped internal surveys to mirror the NSS
– renewed their focus on learning and teaching revisited and improved timetabling systems
– raised structures including building sites, teaching rooms and sports complexes
– revisited their feedback and assessment mechanisms
– organised briefings with students to enlighten them about feedback and assessment, the NSS and its benefits
– replaced subjects, at times personnel, whose NSS scores keep falling
– introduced or empowered various forums and platforms to meet several times in a year to discuss the NSS, among others. Such forums found at nearly all institutions include: NSS forums, student experience action plans, education boards, NSS improvement forums and learning and teaching advisory groups

Across the institutions in the research, other similarities were see in how data was used: for instance comparing scores across schools, holding low scoring schools to account, and comparing with other institutions.

In terns of league tables, depending on where you appear in a league table appears to influence the behaviour of the organisation.

In particular, institutions placed in the top 25% of the league tables appear to have a relaxed view of the NSS. They appear to put particular emphasis on improving the student experience and argue that this automatically triggers a higher satisfaction rate than being ‘obsessed with the NSS’ and improving league table position:

Whereas at the other end of the scale:

In contrast, institutions in the lower 25% of the student satisfaction league tables appear to place particular focus on improving their student satisfaction and subsequently their standings in the league tables.

The main conclusion of the work then is that:

In particular, institutions in the top 25% of league tables
(Universities A and B) appear to prioritise improving the student experience and let the NSS take care of itself, while those in the bottom 25% (Universities C and D) prioritise their NSS league table position and subsequently employ various tactics to promote the surveys.
Despite institutions adopting different approaches to the surveys based on league table positions, institutions generally listen to students’ demands raised in surveys and have responded by instigating various changes including recruiting academic advisers and officers to take charge of the NSS; mapping internal surveys to mirror the NSS; raising structures including building sites and revising their feedback and assessment processes.

What the paper doesn’t consider is the relative ranking of NSS scores by institutions – it is perfectly possible to score well on certain NSS scores, and appear to out perform other institutions on such a single measure, but this may not change institutional behavours which may be set to focus on the NSS position, rather than overall experience.

In other work out recently, from Stephen Gibbons, Eric Neumayer and Richard Perkins writing in the Economics of Education Review “Student satisfaction,league tables and university applications: Evidence from Britain” (S. Gibbons et al./Economics of Education Review 48 (2015)148–164), the authors make the following points:

  • NSS scores have an impact on recruitment applications, but not huge
  • students do not appear to respond directly to quality cues from satisfaction scores
  • students may already have a well developed knowledge about product quality based on perceptions of reputation and prestige
  • student satisfaction and league table position do not have a short term effect on market demand
  • the degree to which quality indicators affect demand is strongly linked to the amount of market competition for a given subject

Finally, in “Applying Models to National Surveys of Undergraduate Science Students: What Affects Ratings of Satisfaction?”  (Educ. Sci. 2013, 3(2), 193-207) by Langan Dunleavey and Fielding of Manchester Metropolitan University, the authors look at what influences the results seen for question 22 – overall satisfaction. We are all familiar with reading through a set of results, with great scores for most of the questions, and sections, but a lower score for this final crucial question, which is the one used in all league tables.

The authors note the year on year consistency of results for individual subjects, noting how comparisons should be made:

Subjects were highly consistent year on year in terms of their relative performance in the satisfaction survey. This has implications for institutional decision-making particularly if subjects are wrongly compared against institutional averages, when comparisons should be made within subject areas (e.g., comparing with national subject averages, although this may be subject to error if courses contain different compositions of learners, for example in terms of ethnicity)

This is consistent with HEFCE advice, and why as an institution we provide sector average scores at JACS3 subject level for comparison.

Interestingly, questions about feedback were the weakest predictors of “Overall Satisfaction” whereas:

The best predictor of student satisfaction nationally for the three years analysed was “The course was well designed and running smoothly” followed by ratings of “Teaching”, “Organisation” and “Support”. This may vary slightly between subjects/institutions, so it is proposed that this type of quantitative approach to contextualising survey metrics can be used to guide institutions in resource allocation to tackle student experience challenges.

So our conclusions on how we approach the next NSS, and perhaps more  importantly NSS2017  could be:

  • carry on listening to students, responding and being seen to respond to surveys
  • make sure we focus on all the measures that make up a league table
  • make sure that courses are well organised and running smoothly
  • don’t expect league table moves to immediately be reflected in increased applications
  • and remember – the student experience is what really matters, not the survey itself.

Wonk’s Delight

And so the waiting is over. Friday finally saw the long awaited release of one the  most anticipated artifacts of 2016. Twitter was inundated with commentary, and major newspapers rushed to add reports to their online editions as the new John Lewis Christmas advert became available.

In other news, the government published its green paper – “Fulfilling out Potential – Teaching Excellence, Social Mobility and Student Choice”.

Much of the content of this was trailed previously in speeches by Jo Johnson, and last week on the Conservative Home website. Plenty of  excellent commentary has already been provided through WonkHE and I’d encourage my colleagues  to read these as well as the paper itself.

Over on WonkHE I’d recommend the following:

David Kernohan writes on the  first TEF’sassessment

Martin McQuillan also focuses on TEF, and considers that universities brought TEF upon themsleves

Andrew McGettigan crucially shows ten things you might miss on a first reading

And in the Times Higher, Jo Williams questions the impact that TEF will have on how universities look at student satisfaction, and how this could affect academic freedoms.

There are some key points for us all to know about the contents of the Green Paper, specifically around the emerging detail of the Teaching Excellence Framework, but also the longer term ambitions for HE in this country. I’ve previously suggested in this blog that the changes are intensely political – they are about creating a deregulatory regime (although that doesn’t quite tally with the regulation that TEF might create), with opening up the “market” further to new entrants, and to allow more rapid shifts in market share. This seems to be driven by a belief that teaching in universities is poor (with little evidence of this) and that institutions need to be able to show better value for money to consumers. The assumption clearly is that HE does work as a market – whereas many including me, would argue that it is not a traditional market, that students are not passive consumers, that they should be active participants in creating their learning, and that universities themselves have created this dumbed down “the customer is right” approach.

Notable in the document is an emphasis on full time undergraduate students Part time barely gets a mention, and although there are nods of acknowledgement occasionally that the sector is more complex than FTUG, this barely affects proposals

Let’s look at the key proposals, on TEF in particular, and what they will mean for us in the short and medium term, remembering that this paper is now open for consultation.

Part A covers teaching excellence, quality and social mobility, and this is probably the section of most interest to most university teaching staff.

Excellence is defined in the paper as:

There is no one broadly accepted definition of “teaching excellence”. In practice it has many interpretations and there are likely to be different ways of measuring it. The Government does not intend to stifle innovation in the sector or restrict institutions’ freedom to choose what is in the best interests of their students. But we do think there is a need to provide greater clarity about what we are looking for and how we intend to measure it in relation to the TEF. Our thinking has been informed by the following principles:
• excellence must incorporate and reflect the diversity of the sector, disciplines and missions – not all students will achieve their best within the same model of teaching;
• excellence is the sum of many factors – focussing on metrics gives an overview, but not the whole picture;
• perceptions of excellence vary between students, institutions and employers;
• excellence is not something achieved easily or without focus, time, challenge and change.

Ultimately there will be different levels of teaching excellence – once the metrics and process have been decided –  however, in the first year, any institution that has a currently satisfactory QA review will qualify for level 1 TEF.

In the short term then, we, along wth nearly everyone else, would “pass” the TEF.

Later there will be up to 4 levels of TEF, and institutions will apply for the various levels. This will be done either on a rolling process or when unis decide that they are ready to apply for a higher level. This is where is starts to be contentious and competitive

A key driver of TEF is to link fees to teaching quality, and so in future the fee cap will likely be set by government for each of the different levels created, so that fees can once again be seen as a differentiator – the current system has led to nearly all traditional providers charging the maximum of £9000, so this has not helped generate the anticipated market behaviours. Whether having 4 values of fees will have any more effect remains to be seen.

The paper proposes that the metrics to be used  initially to be: DLHE, NSS and HESA continuation and retention. You might as well read a league table!

However, importantly, the paper recognises the limitations of metrics alone:

“However, we recognise that these metrics are largely proxies rather than direct measures of quality and learning gain and there are issues around how robust they are. To balance this we propose that the TEF assessment will consider institutional evidence, setting out their evidence for their excellent teaching.”

The institutional evidence might look a lot like the self evaluation submissions for quality activities, In addition to this, there is an expectation of using further new common metrics on engagement with study (including teaching intensity) and learning gain.

Degree classifications and grade inflation cited as a concern, with a nudge towards a move to GPA, but without explicitly asking universities to do so. The numbers of good degrees award nationally has risen dramatically over the last 10 years – partly because of good teaching, partly through a different student attitude to study and the need for a higher classification, but maybe partly through gaming the algorithms to gain league table position

On social mobility, the attainment gap between BME and white students is highlighted, as well as the lack of engagement with disadvantaged white males. These are areas that would be expected to be addressed in the TEF – “Work to improve access and success should have close links with the TEF.”

Other key headlines to be aware are:

  • removal of barriers to entry for new providers, including a possible remval of teh minimum numbers of students.
  • creation of an Office for Students – as part of a bonfire of quangos the work of HEFCE and OFFA would move a new OfS. The OfS would be responsible for assessing quality of teaching through TEF
  • proposed  to constitutional arrangements of Higher Education Corporations which will affect us
  • reducing complexity of research funding, although this will be detailed more in the coming Nurse review

So what do we need to do?

Firstly, a lot of people (everyone who works in the university) should read the Green Paper. This is clearly setting out this government’s ambitions for the future of the sector.

Secondly, a robust response to the consultation is needed – where there are sector concerns, for instance over the detail of TEF, we need to echo and articulate those.

We also need to accept that this is coming our way, so we can start preparing in anticipation now.

Ideally, we should be creating systems and processes for capturing information both qualitative and quantitative, as part of our business operating model, so that we don’t have to do this as a separate later exercise, but build it into standard practice (at the same time as remving other less useful and time consuming processes).

Since we know the metrics to be used in the second year, then we can start benchmarking ourselves against other universities using the data sets in Heidi. We can also do the same with the Unistats data

We can start amending our own portfolio performance review tools to reflect the same information but at subject level. This could be considered gaming the system – but we all know that’s one of the contributory factors in winning in the league table game, and this will be no different.

We can set up course monitoring processes such that detailed data and analysis is readily available to support the work we do on supporting students from widening participation and BME backgrounds – we know we work well in this area, but will want quantitative as well as quantitative evidence

We can start to articulate what does make for excellent teaching so that we are ready to develop our institutional qualitative submission – maybe a review of our ongoing course approval and review process can be carried out to start to capture this now.

Overall, a lot of work needs to be done, firstly to understand the devil in the details of the green paper, to identify how to set up processes and practices to deliver the information needed. The agile organisation will be working out how to do this without creating any extra burden, by removing other activities that are no longer necessary, and making sure that academic staff have the space to focus on the most important work – teaching, and scholarship and research that enables them to be excellent teachers

Most importantly though, is the need for us to keep demonstrating that excellence is not easily measured; that education is not just a product for consumption; that higher education is the  opportunity for people to engage in transformational activity which will provide benefits to themselves as individuals, as well as to broader liberal society.

 

 

 

 

Who are you?

Not the chant at a football ground, but a question we could be asking of ourselves – how well do we know our students?

At an institutional level, or indeed course level we can do two  things. Firstly we could carry out market segmentation that might tell us so much – games designers like to play computer games in their free time for instance. Secondly we can realise benefits through a data-centric approach to supporting teaching. If we can identify for instance, that for a given module, more than 50% of the students taking it entered university with BTEC qualifications, then maybe there are some conclusions for the lecturer to draw in terns of how to deliver and in particular how to assess. As we move into the era of TEF, then this ability to use data to measure outcomes might be come ever more important, with the comments made recently by Jo Johnson on the differential performance of students from WP or BME backgrounds

In terms of having a holistic view of who we teach, are there some generalities we can consider to give a holistic view, and will this help us to understand some of the current trends in higher education?

The dominant narrative in HE is always about 18-21 year-olds who are studying full time. These are the students who, for most institutions, generate most of the income, and also provide the outcomes that are subsequently used in league tables and are most likely to be used in the Teaching Excellence Framework. While recognising the importance of other groups of students, this article focuses on this grouping, since their behaviours tend to dominate policy.

Firstly we can think of how this generation of students can be defined in terms of their attitudes to technology. Born in 1997, they have never used a computer that was not controlled with a mouse or a touch interface. They have been used to having a mobile device all their lives, and a tablet computer for the last 5. So through senior school and public exams, they have been used to having easy access to the web. They can’t even envisage a time when the web wasn’t ubiquitous. They operate in a  world defined by online social media networks and physical storage is a strange concept. Clearly for us to be able to work with these students, we need to have our own high levels of digital capability, and this is a topic I will be returning to in coming weeks.

As well as understanding the technology expectations that these students bring to university, we can also look at some of their motivations. Focusing on young full time undergraduates, then they are what was termed in today’s Observer as  Generation K, or Katniss. (You may need to have teenaged children to understand this reference.)

…the economist and academic Noreena Hertz, who coined the term Generation K (after Katniss) for those born between 1995 and 2002, says that this is a generation riddled with anxiety, distrustful of traditional institutions from government to marriage, and, “like their heroine Katniss Everdeen, [imbued with] a strong sense of what is right and fair”

This generation worry about getting a job and this is understandable after major economic downturn, but the impact it might be having on these students’ attitude to higher education education is that they perceive HE as a means to getting a job. Although this is understandable, within the academy we still have a responsibility to show that higher education is more than that.

Writing in the same newspaper, WiIll Hutton, author of “The State We’re In” and now Master of Hertford College, Oxford, tackles the issues of the freedom to argue and the freedom to be challenged within universities. Hutton is primarily concerned with the retreat of liberalism and the diminution in importance of the idea of a public realm:

This disdain for notions of publicness has created a vacuum occupied by the rise of a libertarian individualism that indulges belief over reason. Non-falsifiable belief systems used only to be the hallmark of ideological communism or religious zealotry. Now, ideas, especially on the right but also to a degree on the left are less and less tested in a public realm by debate, with evidence marshalled to justify them. Instead, they are asserted as valid because the holder believesthem.

This new individualism, alongside the decline of the public, has provoked a mounting tide of, at best, siloed thinking, impervious to criticism, and, at worst, the indulgence of rank prejudice. Thus, on the right, if I feel that Britain is being swamped by immigrants, climate change is bogus, maleness is being overwhelmed by “femininazi” women or the welfare system is transfixed by cheating, then, whatever the facts, my feeling is valid because I hold it. That suffices without proof or evidence. In any case, there will always be some article in a rightwing paper to justify it.

Again, this individualism can be expressed by students – being risk-averse and not willing to be challenged or taken out of their comfort zones leads to behaviours that create “safe spaces” on campus, where we can no longer  hear certain views in case others are offended. In the same way, we might have a group of students who don’t want to be challenged too much in their learning, seeing a degree as a commercial transaction, rather than a transformational experience, the benefits of which may only be realised many years after completion. We ask students every year in the National Student Survey whether they felt their course was intellectually stimulating. We want our students to engage with their discipline, not as passive recipients, but as active scholars providing contribution and challenge. To do this though, we will need to remind ourselves of the tendency to be risk-averse, to want to be treated as a consumer and to be challenged just enough – but not so much it becomes too hard. We would do well to remember that as well as being stimulating, courses are supposed to challenge, be intellectually difficult  and to provide arguments that question individual beliefs.

Finally, this generation of students perceive themselves to be consumers. As argued in her book “Consuming Higher Education”, Joanna Williams of University of Kent argues that this attitude to consumption doesn’t come about just because of the introduction of tuition fees. Instead, Williams argues that we have a generation who have been led into this interpretation of what higher education is about, in part by the behaviours of universities:

“Rather than universities challenging the idea that a degree is an entitlement, institutions instead strengthen this notion. The provision of quantifiable information on contact hours assessment patterns and employment prospects suggests students are correct to to perceive of a degree as a product.”

Individual institutions can’t turn around and say we are having no part of this agenda as we are obliged to publish this information and no doubt will provide more in future. What we can do is explain better what the quantifiable data actually means, and more importantly, what it does not mean.

As we all write our responses to consultations on quality assurance, on the teaching excellence framework, and as we redevelop our own strategy documents in the light of a forthcoming Green Paper, we need to remind ourselves that universities do exist as more than degree factories that are there to produce “satisfied” customers. Understanding more about the backgrounds of our students and recognising the reasons for their consumerist behaviours may help us in the long term to be able to better articulate what the wider benefits of HE are, and the role of universities in liberal society.

 

New Beginnings

Higher Education is currently entering a maelstrom of change – we are waiting with bated breath the publication of the Green Paper, and as suggested previously in this blog much of what is possibly going to change is intensley political and driven much by the treasury. More early hints appeared on the ConservativeHome website this week:

Johnson’s proposals begin with the student experience. Some University teaching is excellent; too much is “execrable”, to borrow a word sometimes used in the department. To help raise the standard, he wants Universities to be rewarded for better teaching. The metrics used will include lower drop-out rates, good graduate outcomes for disadvantaged students, and an improved national student survey. His friends claim that evidence shows students value better teaching above lower fees: there is an reflection here of Nick Hillman’s finding, over at the Higher Education Policy Institute, that they are “less motivated by student issues, like tuition fees, than has often been supposed”.

and as we know there will be changes to QA processes:

If the new inspection body is to be the stick, there will also be a carrot. As George Osborne announced in the Budget, Universities that teach better will be allowed to raise fees in line with inflation from next year. Permitting further rises later has not been ruled out. There will be no shortage of objections to all this. Some Universities don’t want to be challenged by new entrants. There will be questions of detail, such as whether the metrics will work. There will be those of principle, such as whether it is really government’s business to tell the Universities how to conduct theirs.

More locally, we have our own disruption, reflecting on ” Smarter Futures”, and looking at the kind of university we want to be in the future as we enter a period of strategic planning. As part of that we are embarking on updating the  Academic Strategy for the university. So far, this has been discussed in various management groups and committees and is about to be available for wider consultation.

The rest of this blog piece will focus on the changes to the learning and teaching section of that strategy. Smarter Futures asks us on: recruitment and retention: challenging and supporting our students; and finally making sure they can achieve and become employable.

Writing a distinctive strategy is never easy – the aims of all universities to an extent can tend to be the same, as has been reported in a blog article on the Times Higher website this week – where it is suggested that a lack of boldness leads to all strategies looking like a whiter shade of pale.

I know that in writing this, we were urged to make sure we strived for excellence.  At the risk of this becoming a race to mediocrity and an “all shall have prizes” mentality, we’ve tried to highlight the areas we we feel we will be excellent, and those key strategic themes that we need to address.

However, we mustn’t lose sight of what a university is supposed to be – we may be driven by various metrics, both internal and external, but there has to be an underpinning commitment to the ideas of scholarship, for both students and academics, if we are to  attract both groups to be part of our community.

We have to support the notion of higher education as a transformational experience, not just a  transactional marketised service. We should be aware that we are changing lives through what we do, and this should influence the way in which we teach. As Thomas Docherty writes in “For the University, Democracy and the Future of the Insitution“:

Facts are, of course, important in teaching; but, if teaching and learning are to be historical, if they are to be allowed to make a difference to people’s lives in such a way as to give those pupils the autonomy necessary for the assertion of their own authorities, then facts become subservient to experience. It used to be ‘a fact’, for example, that the world was flat; but the experience of circumnavigating the globe changes this ‘fact’, and the experience produces new facts that are themselves, in turn, subject to further modification. If learning is anything, it is a process of transformation and most certainly not of transmission or transfer. It is a process in which I can become something, and in which I can become something other than I am at present. Learning puts me in possession of new facts; and it does this not simply by a process of abstract rationalization, but primarily through historical experience.

Working in an environment where still there are confused manifestations of marketisation and consumer behaviour then we should use our learning and teaching strategy to reinforce the broad goals of HE and scholarship, as well as delivering the right outcomes for external metrics. As Joanna Williams concludes in her book “Consuming Higher Education, Why Learning Can’t be Bought”:

Students and lecturers need to be united in the common goal of developing and interacting with disciplinary knowledge. Too often lecturers and students are presented as being on opposite sides with mutually exclusive interests – lecturers perhaps seeking to protect research time, students to ensure a better service.Learning often depends on the relationships between lecturers and students, and such relationships are prevented from developing if opposition is assumed.

Williams ends with questioning the funding for HE but reiterates the need for “the purpose of education (to be) placed at the heart of the university rather than job training or social inclusion”

As a university committed to success of all of our students, we’ve included sections on employability and inclusion but in the context of offering transformational education.

So here are our six themes, and what we mean by them.  We have tried to identify what we want to do, without reducing ourselves to something less than the ideals of higher education.

1.Developing confident and capable learners

Our learning and teaching has to be able to support all students once they arrive with us, and show them how to become lifelong learners.This is the key which will allow the fullest benefits of higher education to be attained. This is where we tall about students engaging with discipline knowledge as well as wider attributes

2.Providing challenging and supportive learning and teaching

We want to create amazing learning environments – both physical and virtual. Lots of work is already going on to change campus, and we need to keep looking at what others are doing. This week for example I visited the new Diamond building at Sheffield University – an incredible space, open 24/7 for student learning and teaching, including laboratories, workshops, study areas and a lecture theatre. We will be making sure all our new rooms, as aprt of Campus Transformation are designed to allow learning to flourish.

Online we need to transform the way we support learners. We have a  generation of first year undergraduates who  went through secondary school, with BBC iPlayer and a smartphone in their pocket that could access the web no matter where they were. Putting PowerPoint slides (that we read out earlier) into BlackBoard won’t cut it any more. I’ll be writing another short piece this week about how we will be embracing digital in future

3.Raising attainment and achievement.

This has been raised many times before in this blog – we want our students to achieve to their maximum potential. We’ve got lots of work already ongoing this year as part of the Raising Attainment Roadmap, but this remains a key strategic focus. Higher education is meant to be challenging, so we want our students to be challenged and then supported to be able to achieve.

4.Developing Employability

This blog has long argued that higher education is a transformational process, not a simple instrument to lead students into jobs. Nonetheless, improved employabiliity is a key outcome of HE, and it could be argued that many of the other benefits of HE that accrue come about partly through enhanced employment opportunities. As well as all the work already ongoing in employability, the new strategy picks up on three key issues: social and cultural capital; numeracy and data handling, and digital fluency or capability, The first will help students get through the door in the first instance, and the second two will provide enhanced skills in two areas that all graduates need today, based on the demands we see from major employers.

5.Delivering innovative learning and teaching

No L&T strategy would be complete without this – for us this means further development of enquiry and practice based learning, more use of alumni and employers in designing authentic learning tasks and an embedding of all the staff development and sharing opportunities that we have, to make sure we all move forwards.

6. Supporting a diverse population of students

As a modern university, we have a more diverse undergraduate body than many. We also have significant numbers of international students, of part time learners, of mature students. We know that we have one of the highest percentages of WP students in the country. In order to make sure that all of our students are able to attain, we will get better at mining our data to be able to identify trends in differential attainment, and build inclusive practice into all that we do, so that all student groups have an equal chance of success.

As well as these 6 themes, our strategy also talks about what it means to be an academic at Staffordshire University, as well as what it means to be a student.  I hope you’ll read and feedback on the detail once it goes out for consultation.

 

Politics and the TEF

Prior to the general election, I wrote a blog post reviewing the various parties’ views on HE. Following the conservative majority I wrote another piece which concluded with “What is still not clear is how universities might be regulated, how quality mechanisms will operate in future, and how the regulatory and quality regime will be changed to encompass the more diverse range of providers”

Following the various party conferences we now enter a period when we await, with bated breath, the green paper on higher education. For an insight into the Conservative conference, then I recommend “Welcome to the Northern Powerhouse of Cards” by Martin McQuillan of Kingston University

There’s little point in looking at the other parties right now – there is not likely to be an election till 2020, and Labour haven’t identified their position on fees, let alone how they will carry out the role of opposition to the green paper.

The Conservatives are in an interesting situation. Cameron as leader, who has acted as a CEO has already indicated his intention to step down. Hence for everyone else it “eyes on the prize”. As deputy CEO, Osborne has been calling the shots on HE policy, since the Treasury is dictating policy more clearly than any other department. May is setting out her stall, and showing clear opposition to overseas students which will win her no friends in universities. Boris is harrumphing around the margins, and looking more widely Hunt is exerting everyone to work harder.Meanwhile, Javid is happy to drive through large cuts at BIS, and we can expect that many of the organisations that currently work in the HE sector may cease to exist.

It’s into this environment, with his boss supporting 40% cuts to BIS, that Johnson will need to produce  a green paper and ultimately drive legislation through parliament

All of a sudden,this looks threatening to HEFCE. The HEFCE consultation on QA is in tune with government and seems to promote a move to a deregulatory ideology and imply the demise of QAA. More recently though, with questions being asked about whether the remaining amounts of funding could be administered from elsewhere, and the need for a body to run TEF, then HEFCE themselves look more vulnerable.

The Teaching Excellence Framework will clearly be a big part of the green paper. It was a commitment from Osborne (that Treasury driver again) and is detailed in the government’s productivity plan “Fixing the foundations:Creating a more prosperous nation”

Excellence in teaching
4.7 The government will introduce a new Teaching Excellence Framework to sharpen incentives for institutions to provide excellent teaching, as currently exist for research. This will improve the value for money and return on investment for both students and the government, and will contribute to aligning graduate skills and expectations with the needs of employers. The government will consult later this year on how a Teaching Excellence Framework can be developed, including outcome-focussed criteria and metrics. The Teaching Excellence Framework will inform student decision-making, continue to support a high average wage premium for graduates and ensure that students’ hard-won qualifications keep their value over time.
4.8 To support teaching excellence, the government will allow institutions offering high quality teaching to increase their tuition fees in line with inflation from 2017-18, and will consult on the mechanisms to do this. This will reward excellent institutions with higher fee income, while ensuring students get good value from the tuition loans that the government underwrites.

Johnson now needs to steer this through parliament, at the same time as BIS is facing large cuts, and he needs to produce something that will work, both as a fix in the short term, and as a longer term evaluation of teaching.

To be able to have variable fees from 2017-18 will mean measures in place during the current academic year. Inevitably this will be based on existing measures – NSS, Hesa returns, DLHE initially.

Longer term though, then a new set of measures will come in which will provide challenges to the sector, and to individual institutions. From the Times Higher Johnson has made it clear how he would like the metrics to be set up:

Widening participation and access will be intimately linked to the TEF. One of the core metrics we envisage using in the TEF will be the progress and the value add [for] students from disadvantaged backgrounds, measuring it for example in terms of their retention and completion rates. And their [universities’] success in moving students on to either further study or graduate work.

On having an impact on further marketisation, then Johnson says:

the system should “not only have the capacity for more rapid market entry, but we [should] have the capacity for more rapid market share shifts between universities than we have hitherto seen in the sector”.

and  that

he wanted a system where “market share can shift towards where teaching quality really resides. Our teaching excellence framework will be an important signal to students of where quality resides, discipline by discipline, institution by institution.”

He’s asking an awful lot from a set of metrics that are not yet defined, and that will have numerous questions raised by many in the sector.

In the meantime, what can individuals and institutions do?

Firstly there is the opportunity to respond to the government’s inquiry into assessing the quality of HE, which asks specific questions such as:

  • .What should be the objectives of a Teaching Excellence Framework (‘TEF’)?
  • What are the institutional behaviours a TEF should drive? How can a system be designed to avoid unintended consequences?
  • What should be the relationship between the TEF and fee level?

Secondly we can  start looking at the various measures of value added or learning gain for different groups of students. HEFCE are already supporting a range of projects involving over 70 institutions to look at learning gain.

One of the unintended consequences that TEF might bring about is a gaming of the system. I’m not suggesting that data returns that feed into league tables are inaccurate, but one part of a successful league table result is a set of carefully constructed data returns. It’s equally likely that it will be possible to do something similar with any TEF submission, so all institutions will learn very quickly how to report data in the best possible way

Finally, recognising that TEF will be used to drive rapid shifts in market share (a euphemism?) then we will all need to get very good, not only at supporting the widest range of students, but also at understanding how the metrics apply to us, and how we can build internal systems to replicate them.

 

 

 

The Midnight Bike Ride

Once again a small number of hardy souls from the University will be taking part in the annual British Heart Foundation Midnight Bike Ride from Manchester to Blackpool, over the night of 26th-27th September.

If you’d like to join us, there is still time to book into the event via the BHF website – it’s not a race, so endurance is more important than speed, just let me know if you’re coming along and we can meet up at the beginning. And maybe the end.

Even better, it would be great if you could sponsor one of us – here is the link to my JustGiving page

Bike Ride_1018789795

(from visitblackpool.com).

I promise not to flood this blog next week with pictures of me or other senior colleagues dressed in lycra……

Differences in Degree Outcomes

New from HEFCE this week, a report on “Differences in Degree Outcomes:the Effect of Subject and Student Characteristics“, which looks at the outcomes of students who graduated in 2013-14. Some of this data I have previously reported when looking at HESA data on the impact of ethnicity on degree outcomes for the previous year.

The results of the HEFCE survey are not startling – they almost reinforce things that we already know in terms of what factors have an impact on achievement: the challenge now is to learn how to address each of these, and with the recent comments by the new universities minister on widening participation, and our own commitment to supporting a diverse population of students then awareness of these trends and how we then tackle them will be crucial for success of individuals and of the institution.

HEFCE considered the following variables when looking at the differences in outcomes:

  • age
  • disability status
  • ethnicity
  • The Participation of Local Areas measure (important for high WP populations)
  • sex
  • subject of study
  • prior attainment (in terms of qualifications held on entry to higher education)
  • previous school type
  • institution attended

The interesting part of the analysis is not the differences in outcomes that can be seen, but how much these differences can or cannot be explained by the influence of other factors.

Subject

Certain subjects are more likely to award 1sts/2(i)s, and the table below represents those subject we offer at Staffordshire – it will be interesting to compare our recent results with those for the sector by subject.

Subject % first or upper second % first
Subjects allied to medicine 69% 24%
Biological sciences 70% 18%
Physical sciences 73% 25%
Mathematical sciences 73% 35%
Computer science 66% 28%
Engineering and technology 74% 30%
Social studies 73% 16%
Law 69% 12%
Business and administrative studies 71% 21%
Mass communication and documentation 75% 15%
Historical and philosophical studies 82% 19%
Creative arts and design 72% 21%
Education 68% 18%
Combined 60% 16%

I always thought it was apocryphal that law didn’t award firsts – across the sector it would appear to be true!

Entry Tariff

On entry tariff, there is a clear relationship – higher entry leads to higher numbers of good degrees, which can also be seen when looking at league table data. This is one of the reasons that the Guardian league table uses a “value added” measure which seeks to adjust for entry tariff..

hefce1st1

 

Mode of Study

In general, part time students have worse outcomes compared with full time. Even adjusting for variations on entry tariff, part time students have worse outcomes than full time.

Age

The raw data shows that young students are 11 percentage points more likely to gain a good degree compared with mature entrants.

Gender

Across all entry tariffs, women are more likely to gain good degrees than men.

Disability

Graduates with a disability are slightly less likely to gain a good degree than those without a declared disability.

Ethnicity

This is the area with the biggest gap. 76% of white students gain a good degree, compared to 60% of black and minority ethnic students.

Even allowing for other factors, the unexplained gap is still equivalent to 15%.

Previous School

In most cases students from state schools outperform those from independent schools.

Neighbourhood HE Participation

Students coming from neighbourhoods with the highest rates of HE participation also gain the highest numbers of good degrees.

Implications

The recent speech by Jo Johnson referred to the importance of universities in driving social mobility and the sector’s work in widening participation.

This data provides further information that could be used to justify the costs of supporting WP in universities, and for focusing on trying to close gaps in attainment.

Much focus is given to looking at the data provided by UCAS but to understand how well the sector and individual universities are performing in terms of closing these gaps, then much fuller datasets need to be considered, taking into account retention and progression and ultimately employment – even if all our students gain the degrees they deserve, but still fail to progress into appropriate graduate roles, then social mobility isn’t realisable for everyone.

As we move into a potential quality regime that could be metrics based, together with a Teaching Excellence Framework, which will certainly use a variety of metrics (possibly including learning gain), then there will be plenty of work to be done in generating data and analysing it..

However, the focus also has to go beyond analysing data. How can we use it to understand our students both as individuals and as cohorts? How can we use data to support our staff better in teaching and assessing their students? Finally, how can we learn to change practices and behaviours based on evidence?

 

 

 

Good University Guide 2016

The final big league of the year was published today – the Good University Guide, which comes from the Sunday Times.

Details of methodology and subject tables are available on the Sunday Times website, behind a paywall, so won’t be discussed here, however this iteration of this guide does use the results of REF2014, and unlike the other major guides published this year uses the NSS data published last month.

Big winners this year are Harper Adams, Bath Spa, Manchester Met while biggest drops are from Arts Bournemouth, Chester, Arts London, Cardiff Met, BCU, Cumbria.

The great news for Staffordshire is that we have risen a further 6 places, which means a rise in all the league tables this year.

Our individual data shows:

Performance Measure Score Rank
Teaching Quality 81.80% 49=
Student Experience 82.80% 80=
Research Quality 16.50% 55
UCAS Entry Points 274 118
Graduate Prospects 58.40% 111
Firsts and 2(i)s 63.20% 98=
Completion Rate 78.40% 115
Student-Staff Ratio 16.8:1 59=
Services/Facilties Spend £1,620 81

Notable results for us then are our ranking in research – although we submitted a relatively small number of academics, the editorial in the paper does comment on our increased size and scope of research, noting our best results were in Sport and Exercise and with psychology scoring high for external impact.

The impact of our teaching quality score is pleasing,and if we can improve this together with the overall student experience score, we will see further improvements in this guide next year.

The new work we are doing this year to enhance student employability together with our Roadmap for Raising Attainment, both of which will be reflected in the new Learning and Teaching Strategy, will lead to further improvements in good degrees and graduate prospects.

As alluded to earlier, the tables are behind a paywall, but parts can be constructed from the press releases from the Sunday Times as below:

Name Ranking 2016 Ranking 2015 2015 National student survey Teaching excellence 2015 National student survey Student experience Graduate prospects Completion rate
(%) (%) (% in professional jobs or graduate-level study) (%)
Cambridge 1 1= 83.8 86.3 89.3 98.4
Oxford 2 1= 83.1 86.8 87.1 96.3
Imperial College 3 4 79.8 87.8 91.1 96.5
St Andrews 4 3 83.2 86.8 83.3 95.3
Durham 5 6 81.9 86.7 84.4 96.6
Warwick 6 8 79.6 85 79.8 96.7
Exeter 7 7 82.6 87.7 79.8 95.7
Surrey 8 11 86.9 90.3 78.8 92.2
LSE 9 5 72.1 78.4 78.5 94.8
University College London 10 9 74.2 81.3 83.1 94.6
Lancaster 11 12 82.3 85.4 82.5 93.5
Bath 12 10 82.7 87 85.2 96.1
Loughborough 13 13 84.5 89.3 83.7 93.2
Leeds 14 17 83.7 88 78.4 93.5
York 15 16 81.7 86.6 76 94.3
Southampton 16 18 79.3 86.5 78.1 92.5
Birmingham 17 15 80.8 84.2 86.7 94.8
East Anglia 18 14 83.2 88.8 70.3 91.9
Sussex 19 25 78.6 85 84.1 92.9
Bristol 20 19 75.5 81.5 79.6 96.6
Sheffield 21 21 81.3 87.2 75.7 94.4
Edinburgh 22 22= 74.5 82.4 78.6 91.3
Newcastle 23= 22= 82 88.4 79.1 95.1
Kent 23= 30 81.5 85.4 76.7 90.7
Nottingham 25 22= 79.5 83.9 81.3 93.2
Glasgow 26 26 80 86.9 79.3 88.4
King’s College London 27 29 73.9 79.7 85.7 92.8
Leicester 28= 20 77.5 84.4 72.1 92.5
Manchester 28= 28 79 84.7 78.5 92.9
Aston 30 34 83.3 87.9 78.8 90.9
Reading 32 33 80.5 85.8 70.3 92.3
Cardiff 33 27 80.7 86 80.1 93.4
Queen Mary 34 37 80.5 83.3 73.3 91.2
Essex 35 32 83.7 88 64.1 85.6
Royal Holloway 36 34 82.6 84.1 62.7 92.3
Dundee 37 45 84.4 87.1 80 86
Liverpool 38= 36 77.9 83.4 76.1 91.3
Heriot-Watt 38= 41 80.8 84.2 78.1 87.4
Buckingham 38= 48 88 88.4 83.4 86.3
City 41= 46 82.7 85.6 78.9 86
Swansea 41= 43 82.6 86.5 81.4 89.7
Keele 43 40 87 90.2 76.1 90.8
Soas 44 31 75.2 80.8 68.3 80.7
Aberdeen 45 44 77.4 83.7 76.2 84.1
Strathclyde 46 39 76.2 85.6 72 87.6
Coventry 47 42 87.6 89.3 74.2 85.8
St George’s 48 78.7 81.6 93.4 92.7
Harper Adams 49 63 82.6 89.3 73.3 90.6
Stirling 50 53 78.7 82.3 73.3 85.7
Royal Agricultural 51 79.3 85.6 69.7 96.3
Bangor 52 50 85.8 87.8 67.7 81.8
De Montfort 53 54 82.4 84.6 76.9 86.5
Nottingham Trent 54 52 83.6 85.8 67.6 89.6
Oxford Brookes 55 49 83.2 85.7 69.2 89.4
Falmouth 56 51 83.7 83.5 74.5 85.4
Bath Spa 58 70 85.8 85.8 55.1 89.9
Portsmouth 59 57 83.4 85.9 66.9 87.6
Brunel 60 47 78.2 83.8 63.4 87.7
Norwich Arts 61 83.8 83.5 63.4 88.7
Lincoln 62= 60 81.1 84.6 70.7 87.6
Creative Arts 62= 74 82.7 81.5 52 85.8
Northumbria 64= 66 82.9 85.1 66.3 87.6
Winchester 64= 61 84.2 85 60.7 85.2
Goldsmiths 66 55 76.6 76.3 56 82.4
Hull 67 58 80 83.8 66.7 86
Edge Hill 68 72 83.2 83.5 63.8 86.2
Huddersfield 69= 77= 82.2 84 74.1 83
Robert Gordon 69= 64 80.6 83.5 83.1 83.4
Chichester 69= 65 84 85.3 57.5 89.9
Sheffield Hallam 72 62 80.9 83.9 64.7 86.9
West of England 73 68 80 82.5 70.8 84.9
Liverpool John Moores 74 71 81.6 85 63.3 84.2
Bradford 75 76 78.8 84.4 75.2 83.8
Hertfordshire 76 79 78.6 82.8 75.3 86
Manchester Metropolitan 77 89 81 82.2 63 84.4
Roehampton 78 73 78 79.9 60.9 81.7
Liverpool Hope 79= 86.8 86.4 53.9 82.8
Aberystwyth 79= 93 78.4 80.4 62.5 89.3
Arts Bournemouth 81 59 81.4 80 61.4 92.3
Northampton 82= 56 82.3 84.1 60.7 85
Bournemouth 82= 88 75.2 78.8 66.4 86
Derby 84 81 84.4 85.3 60 83.7
Middlesex 85= 75 78.7 81.8 64.9 77.8
Plymouth 85= 80 82.3 84.1 60.2 84.8
Chester 87 67 82.7 83.7 63.6 80.5
Gloucestershire 88 83 79.8 82.6 55.7 86.3
York St John 89 87 82.4 83 65.3 90.4
Brighton 90 82 78.6 80.9 66.9 86.9
Leeds Trinity 91 91 83.5 82.3 65.8 81.7
Central Lancashire 92 77= 80.4 83.5 62.4 81.6
Edinburgh Napier 93 97 80.2 83.7 69.1 81.2
Glasgow Caledonian 94 84 77 82.5 70.2 83.2
Staffordshire 95 101 81.8 82.8 58.4 78.4
Queen Margaret, Edinburgh 96 86 78.9 82.3 59.6 82.4
Abertay 97 106 80 83.1 65.6 75.5
Salford 98 105 80.1 80.9 59.5 79.5
Arts London 99 85 75.9 74.5 59.2 85.5
St Mary’s, Twickenham 100= 100 80.5 84.5 66.7 83.7
Worcester 100= 107 81.4 84.5 63.9 85.8
Teesside 102 94 82.9 84.3 59.8 80.8
Cardiff Metropolitan 103 90 79.1 81.8 59.8 82.2
Sunderland 104 99 82.5 84.2 62.3 81.4
Birmingham City 105 91 78 79.3 64.8 84.4
Greenwich 106 98 79.2 82.4 58.2 84.7
Canterbury Christ Church 107 96 80.7 81.9 57.8 82.3
Anglia Ruskin 108 110 82.5 83.9 65 79.3
Buckinghamshire New 109 116 81.5 81.1 57.6 82.5
Bedfordshire 110 108 80.4 82.7 58.3 80.1
Kingston 111 117 76.1 79.9 60.7 82.2
Bishop Grosseteste 112= 102 80.9 80.6 69.1 90
South Wales 112= 114 77.5 78.3 59 81.7
Leeds Beckett 114 111 78.5 82.7 58.5 78.8
Westminster 115= 112 72.9 80.6 55.1 80.4
Southampton Solent 115= 115 79.5 82.1 54.6 76.8
Newman 115= 104 83.1 84.6 54.6 73.3
West of Scotland 118 118 81.8 81.8 65.7 70
Cumbria 119 95 76.9 77.6 64.9 85.7
London South Bank 120 122 77 81.3 67.9 74.6
West London 121 109 76.9 77.5 60.5 73.9
Glynd?r 122 113 80.5 79.3 66.3 75.8
Bolton 123= 120 81.6 80.8 60.1 71.2
St Mark & St John, Plymouth 123= 102= 76.3 77.3 60.4 82.6
London Met 125 123 76 78.6 47.7 75.3
Highlands and Islands 126 121 78.8 76.5 56 68.6
East London 127 119 75 78.5 45.6 67.5

Jo Johnson’s Speech to UUK

For the second time, our Minister with responsibility for universities makes a speech on HE, this time to Universities UK and an assembled throng of Vice Chancellors. Our own VC’s impressions will be in his blog next week.

The full text of the speech is here, Higher education: fulfilling our potential, but are there any hints of what is to come?

there is considerable unfinished business and the green paper will seek views on the changes the government believes will be necessary to ensure that higher education continues to be a great national success story in the years to come

We can expect to see the Green Paper that will address this unfinished business in the current session of parliament.

The next section of the speech is about “teaching at the heart of the system”, which is different from the previous mantra of “students at the heart of the system” which has subsequently become a strapline in many a university mission statement.

At the centre of this vision are the young people contemplating their futures in a world where no one owes them a living, where they must depend on their wits and drive to survive.

Well-equipped students ready to contribute to society and to businesses keen to employ increasing numbers of skilled graduates

So once again is higher education being seen as a transactional good, rather than a transformational experience, just for young people to enter the employment market, red in tooth and claw? Even though a majority of students may be young, we need to remind ourselves that HE is not just for young people, and not just to deliver training for employment.

Moving on to talk about the proposed Teaching Excellence Framework (however with no proposals or suggestions at this stage), Johnson refers to teaching staff who go the extra mile providing feedback and email replies at weekends, as well as those who think:

we’ll award you the degree as the hoped-for job ticket in return for compliance with minimal academic requirements and due receipt of fees’

I’m sure plenty will take issue with this – although in a sector that employs over 194,000  academic staff (HESA data 2013-14), there must be a few individuals in this category although we wouldn’t expect an institution to behave overall in this way. The speech goes on to talk about the variability of student experience and:

There is extraordinary teaching that deserves greater recognition. And there is lamentable teaching that must be driven out of our system. It damages the reputation of UK higher education and I am determined to address it.

Is the comment on lamentable teaching about individuals or about entire institutions? The TEF is  to  assess an institution, not individual performances. The answer to that might provide an answer of whether solutions are to be centrally driven or can be dealt with through internal management processes and enhancement activities.

Johnson suggests that more information will be provided in future about kind of teaching that students will receive. Having been to a number of open days in other institutions recently I’v e been struck by the willingness to brag about high contact hours, with little reference to what they actually involve.

The new framework will “reward universities that do most to stretch young – and also not so young – minds”. At last a recognition that not all students are 18 years old – but the devil will be in the detail of this: how will the framework actually measure the amount of stretch? This may lead us to look at ideas of learning gain and value-added, but can this be put in place quickly enough for TEF to be ready for 2017?

Johnson asks for there to be  a shift in how we think about teaching – and in an institution that is focused more on teaching than research, then this should be good news.

Widening Participation appears as a major part of the speech – and at a time when cuts to the BIS budget are due, and many commentators suggesting that the Student Opportunity Fund which supports WP activities is likely to be under threat, then we have a series of laudable ideas but which will need money to support.

There is a clear commitment to the use of HE as a driver of social mobility with particular emphasis given to participation rates of working class white boys, and students from BME backgrounds.

Plenty of reference is made to the data provided by UCAS, which will provide the necessary information on recruitment into universities, and a comment is made about linking this to other datasets. If we are serious about being able to understand how well WP activities are working, then we need to look not just at UCAS and recruitment data, but also at information on retention, progression and achievement – only when we can see significant reductions in the differences between different groups in all of the relevant factors would we be able to say that we are making steps to increase social mobility. Much of this information is already held within universities, and plenty of them provide analysis of this to explain their outcomes.

On alternative providers, Johnson recognises that not everyone wants to study a 3 year degree, and praises the alternative provider sector in being able to offer other provision, through validation arrangements with existing universities, but suggests that the current process “stifles competition, innovation and student choice, which is why we will consult on alternative options for new providers if they do not want to go down the current validation route.”

On regulation, Johnson states that this a a deregulatory government, and the recent consultation on QA from HEFCE seemed to push towards a system of no central agency, but with increased role of governance (similar to the rise in the school sector of  Academies and removal of Local Authority control?).

So we need a simpler, less bureaucratic and less expensive system of regulation. A system that explicitly champions the student, employer and taxpayer interest in ensuring value for their investment in education and requires transparency from providers so that they can be held accountable for it. One that protects institutional autonomy and academic freedom and maintains the highest quality of higher education, safeguarding the strong international reputation of English universities

And this is clealry talking about the role of HEFCE rather than universities in countries with devolved administrations.

Overall – will this please the sector?

  • A commitment to great teaching won’t be argued with – the mechanisms of assessing it will be.
  • The change in regulation for alternative providers might be seen as a threat to some institutions (probably only those in the bottom quartile of league tables, or current FE providers of HE)
  • The focus on widening participation should be welcomed – provided that funding and full data analysis is part of the deal.

As ever, the details have yet to merge and plenty of other commentators will have more to say on this speech than I, and we will all eagerly await the Green Paper, and in particular the plans for a Teaching Excellence Framework and a renewed focus on widening participation.