Listening to Students

In a moment of serendipity, in the last week I started looking at some online solutions for gathering and sharing feedback and evaluation of modules by students. In the same week, articles appeared in the press on the involvement of students in providing feedback, and how, when and why we listen to them.

feedback(from http://learningspy.wordpress.com/2012/02/20/feedback-its-better-to-receive-than-to-give/)

In the first instance, the Times Higher provided a piece by Martin McQuillan of Kingston University, on selective hearing.

He starts with:

In recent months universities up and down the country have been engaged in a frenzy of activity in an effort to ensure a favourable report from their students in the National Student Survey. Slides are presented to students and articles written for campus newspapers detailing improvements to teaching and services made in response to student feedback; survey responses are solicited via emails and phone calls; course reps, societies and clubs are asked to encourage undergraduates to take part; and participants are offered the chance to win book tokens, printer credit, graduation ball tickets, iPads and Kindles.

So far so familiar. But the key point is in the next line where he emhasises that the universities who receive the most positive feedback (and possibly attain elevated league table positions because of this) are those who listen to student feedback throughout the year, not just in the time period of the NSS.

In terms of us at Staffordshire, then we will be looking to reinforce this in two ways – firstly by changing the way in which we advertise NSS to students – the emphasis in future will be on a continuous dialogue between school managers and their student bodies, starting in Welcome Week. Secondly, a more focused module evaluation system, with the right processes sitting behind it (ie not just a technological solution to get people to fill in questionnaires) will also provide a basis for dialogue.

Of course this is all very well when things are going right. as McQuillan says in the article:

However, universities are not as good at listening to the student body when it questions management decisions or criticises government policy.

On such occasions, dissenting voices are not considered co-creators of an academic community, but are instead frequently dismissed as part of a minority of troublemakers. In the rare event of student criticism that is accompanied by open displays of dissent, such as occupations and demonstrations, it is usually met with the full rigour of institutional procedure and more often than not criminal law.

The challenge then, is to accept that if we are regarding our students as co-creators (and it appears in our Academic Strategy) then we have to be prepared to deal with tough questions accordingly.

Balanced against this though, is how well prepared are students to act as co-creators, and to provide the right kind of feedback. In a linked article, Joanna Williams of and Jennie Bristow of the University of Kent argue that the student voice has been “tamed, domesticated and institutionalised”.

Noting that the rebelliousness of students seen in previous generations, has for the most part disappeared, or is swiftly put down by forces of law, they note that the student voice:

 is encapsulated in the image of the good student who gives feedback when asked, contributes to staff-student liaison committees and makes only realistic suggestions that confirm a consensus. Regardless of the type of institution attended or the diversity of the student body within an institution, the student voice proves itself remarkable in its homogeneity. Demands for assessed work to be returned more quickly and with better feedback echo around every university in the country.

As I suggested above, I think it key that dialogue occurs between the two parties. Williams and Bristow write:

Lecturers who do not take heed of the student voice risk a poor departmental performance in the National Student Survey, and a low ranking for their university in institutional league tables. The following year, “customers” may not be so forthcoming.

This can make it difficult for academics to challenge the student voice, resulting in what Duna Sabri, visiting research fellow at King’s College London, has termed “the sacralisation of the discourse of ‘the student experience’”.

The first paragraph is true – and if customer” or students do not come, then subject areas might be under threat. But rather than dismiss the student voice, again the need is to recognise the benefit of engagement. Williams and Bristow do usefully identify that there is work to do by universities, in developing their students so that:

For students, the aspiration to be the intellectual equals of their lecturers and critically engaged in the search for new knowledge or the reinterpretation of existing knowledge is entirely laudable. But this should be a privilege students earn after having engaged in an intellectual struggle to master the foundations of a discipline.

Student voice however does need to go beyond just the challenge of the discipline. Students are paying or borrowing substantial sums of money to be at our universities, and rightly expect certain standards of provision to exist, however they won’t know everything (although I’ve met plenty of undergraduates who think they do. And not a small number of staff too).

thinkers_cartoon

(from http://www.joebower.org/2014/04/the-problem-with-getting-students-to.html)

In a separate piece in the Guardian (student feedback is a waste of everyone’s time) , an anonymous academic rails against the way in which students are able to comment on his or her lecturing, on the content of courses and how enagaging the lectures are.

Comments included: “They are not experts in the field and are not well-placed to assess the relative merits of a course.”, “The course content reflects academic research and theory on the subject and is not up for discussion.”

In more detail:

Student feedback is a waste of everyone’s time. It’s disingenuous to ask students constantly to complete feedback forms that will probably do little more than fester on a hard drive until someone needs to cherry-pick the good comments for annual reports or applications for promotion.

It’s also completely redundant: universities appoint external examiners who review course content and marking annually, and there are regular peer observations by colleagues. These ensure professional standards are maintained. As to the question of whether students are satisfied, I’d rather they had an education.

Really? Feedback from students can be powerful and useful, if used properly. And teh reference to external examiners is a red herring – their role is to assure academic standards, not the overall experience of students. Peer observation has a place, provided it is genuinely developmental, but does rely on our peers being prepared to challenge us.

 

The perceived arrogance in the piece prompted debate on Twitter and in the comments under the article are well worth reading. All I would say, is that this is an approach to take that is guaranteed to disengage students. We’re not here to make it difficult for the sake of it. The subject might be difficult, but we want students to have an experience that allows then to engage with their learning and to become schooled in the discipline, and when necessary  have that conversation with us, with respect on both sides, to allow us to improve it.

 

20 ways to improve your university ranking

This week, THE reproduced an article from 3 years ago on how to improve university rankings. The article is mainly aimed at research universities who want to go up in the various world rankings, but since I do quite a bit of work on our analysis of and approach to interpretation of UK league tables, I thought it would be interesting to have look at the various suggestions.

Many of them are around leadership and management, and maybe don’t apply in exactly the same way in a teaching-led institution. However, the way in which we choose to run a university will have an implication on the insitution’s outputs, as we need to create environments that allow staff and students to thrive.

1. To change a university, you need to change people’s incentives

2. To attract the best faculty, you need the best leaders

3. Control quality through hiring panels

4. Hire the best

5. Know the talent list and congratulate people

6. No pain, no gain

7. Too much change, no gain

8. Pay a top salary if you want the right department head

9. Incentivise raising research money

10. Cut the red tape and reduce the number of committees

11. As a leader, be accessible

12. Clarify the relationship between administrative and academic staff

13. Start to train scholars in management when they are young

14. Pick your board or council members because – and only because – they are good for the university, and then educate them

15. Tell Government ‘No!’

16. Give staff food for their tummies as well as thought

17. Hire a scholar as leader

18. Make sure the leader stays at least five years – and preferably more

19. Give the leader plenty of power (or don’t bother hiring one)

20. Let the leader pick his or her own top team

 

It’s interesting to read the detail behind each of these headlines, and see how many we could tick. I’ve done it, and I’m not going to write my answer here.

However I am very mindful of 6 and 7. If we carry on as we have before we will not see an improvement in our performance and the reflected outcome in league tables. So some changes are needed, but the challenge is it make sure that we focus on the right changes, and don’t drown in a sea of initiatives. We need to communicate change in a way that everyone knows what the rationale is, that everything is being driven to improve individual attainment and institutional success.

Considering factor 16 though, I can confidently say that the coffee and muffin selections have improved massively over the years.

A long discussion about change at a recent Heads of School/Associate Deans meeting focused on how sticky or carroty we needed to be….

carrot_on_stic_450

 

Staffordshire University International Partnerships Conference

This week we hosted our first ever international partnerships conference, with delegates from all over the world, from our overseas partners.

The aim was to build relationships between our partners, and to discuss issues affecting transnational education, particularly quality assurance and enhancement. It was personally a great opportunity to meet up with colleagues and old friends I have worked with over the years, frequently through reviews, valdations and exam boards.

My keynote was around improving student outcomes and engagement, and I placed considerable emphasis on league tables, how they are constructed, and how we perform in them. League table position is a concern to us as well as to international partners, but hopefully I managed to shed some light on how they work, what we are doing to address the results, and where we actually do OK.

My slides can be seen on Slideshare, and meanwhile, here are some of the Twitter questions that came up during the talk.

 

20131122-204735.jpg

20131122-204743.jpg

20131122-204752.jpg

20131122-204759.jpg

Annual Survey of HE Leaders

The fifth annual survey of leaders of HEIs in the UK (Charting a winning course – How student experiences will shape the future of higher education) was published today by PA consulting .

The report is summarised as:

“This year’s survey report records the beginnings of a sea-change in the strategic priorities of leaders across the HE system. In place of their historical obsession with the outlook for Government policy and funding, leaders appear to have switched their focus to the competitive battle for fee-paying students and the imperative to offer attractive and rewarding learning experiences. This imperative is driven by the effective demise of grant funding for teaching, coupled with slowing and potentially falling student numbers and increased competition from alternative routes to higher learning.

Sector leaders are unimpressed by predictions that online alternatives will sweep away conventional providers of higher education, expressing confidence in the resilience of the established system to embrace and adapt to new ways of working. Nonetheless, our respondents are united in expecting the emergence of a very different HE system, characterised by a diversity of tailored and student-centred learning experiences delivered through a patchwork of provider partnerships, collaborations and alliances.

Our survey reveals a widespread expectation that not all current providers will survive this disruption, with predictions of institutional failures. There are however few signs of this actually happening. The more likely outcome, in our view, is a radical restructuring of relationships and ventures within and between providers, rather than a widespread shake-out of institutions. “

The greatest worry expressed by HE leaders is around future student demand – not really surprising considering a number of factors such as Changing demographic of UK population, with a reduction of 18yer olds for the next few years and the perceived lack of welcome from the UK towards international students. Over 90% were worried about the decline in UK/EU numbers of postgraduate students, and 80% about international postgraduates. As the report states:

“It is becoming apparent, as the market data increasingly validate these worries, that real competition for students of all types is becoming the major force for change in higher education”

This means that student experience is becoming increasingly important, and an area where universities will seek to differentiate themselves.

90% of respondents said that improving the student experience proposition was among their top three strategic priorities

“Strategic motivations for this priority were, however, polarised between those leaders who regard improved student experiences as primarily a driver of institutional standing (for example as factors in league table ratings or as a source of market distinctiveness) and those who are more concerned to improve students’ learning outcomes and/or employment prospects.”

This is an interesting split.  There is a real danger in being driven just by league tables, and forgetting that they are simply a mirror held up to us to see a reflection of our performance.  While improving our league table position is important, my view is that our focus has to be on improving student experience and outcomes, and allowing this to drive the league table.

To improve experience, many respondents indicated that increased contact with and access to academic staff would be desirable, whilst recognising the cost implications. In my view, this is where an L&T strategy could be designed which would ensure that contact was relevant, and significantly higher for the earlier levels of an award, with a subsequent decrease in the later years, with better use of technology supported learning for the more experienced learners.

It’s interesting to look at the graph below showing the factors that leaders felt inhibited improvements to student experience – cost implications and government or funding policies figure highly.

PA rept4

(from Charting a winning course – How student experiences will shape the future of higher education)

 

Another interesting finding of the survey was that, after all the hype about MOOCs (previously written about ad nauseam on this blog), many university leaders do not see them as a disruption which could remove established models of higher education. Many did think that the new technologies could lead to new forms of blended learning and blended pathways. Regular readers will know that this would be my take – unless we are all wrong and an avalanche really is coming.

The survey this year has suggested that “HE leaders have little expectation that Government or ‘official’ sector bodies will be central to their future success” and:

“None regarded Government departments or agencies as prime sources of innovative thinking or stimulus for change regarding student experiences (most institutions look first to their own staff and students for new thinking in this area). In this, as in many other regards, it is increasingly apparent that HE leaders no longer see themselves as responsible for delivering public education policies, and are looking to grow their institutions’ futures in a very different, learner-centred market environment.”

Our own university plan reflects this in its focus on partnership with the various key stakeholder groups – of which government is not one.

The outcome of the survey shows the prevailing neo-liberal view of higher education where student outcomes are measured very much in terms of the benefit to the individual student and their individual employment prospects, rather than the benefit that may be gained by society as a whole through their education.

Most HE leaders surveyed though that the sector was going to change in size and shape, with mergers and closures as part of the change. This is similar to previous survey results – which seem to say,” yes there’ll be closures, but it’ll be someone else.” The most anticipated change is in multi-institution partnership, alliances and networks.

PA rept6

 

(from Charting a winning course – How student experiences will shape the future of higher education)

Overall – the two most interesting things in this report for me are: the emphasis on student experience and how that could be differentiate between institutions; and the view of how some technologies will not be the disruption that others believe.

 

People and Planet Green League Table 2013

Well at this time of year, league tables come thick and fast. I’m not going to cite the one on studentbeans.com which ranks universities on the “activity” hem hem of their students.

20130611-084744.jpg

Today sees the announcement of the latest Green league table from People and Planet, an area where we as a university make a significant commitment.

According to the Guardian:

Taking the No. 1 spot in this year’s People & Planet Green League is Manchester Met, which jumps from 10th place last year. Coming just half a point behind is Plymouth, in second place for the second year in a row. Plymouth scores full points for every policy measure apart from ethical investment. Greenwich, top last year, slips to 6th place, but is still just 3.5 points behind the leader. The most-improved is Sheffield University, which jumps 63 places to 56th. Its giant leap is largely thanks to a strong new sustainable food policy, increased environmental staff capacity and the introduction of ambitious carbon-reduction targets.

Staffordshire has risen 15 places to 16th in the table,which is excellent news.

Looking at the detailed results for our university, then the weakest area is in integration of sustainability issues into the curriculum. We can argue that we deal with this under global citizenship, as part of the Staffordshire Graduate attributes, but maybe we need to offer more guidance to staff and students on how this is embedded.

Guardian University Guide 2014

The big news of the week was the publication of the Guardian league table, for students entering HE in 2014. This table is important for Staffordshire University, as improvement in this is one of the KPIs in our university plan. The Guardian table is the one we focus on, as we believe it most closely represents how we see ourselves as a university.

The good news is that we improved 4 places on last year, up to 92nd. Still a long way to get to where we were a few years ago, but we have hopefully now reversed the downward trend.

The rise in overall position appears to be attributable to:

  • improvements in the three NSS factors;
  • a reduction in student: staff ratio and
  • increasing entry tariff.

Career prospects are lower than last year, as we also saw in the Complete University Guide, so this is not an unexpected result. Had this not decreased, then we could have been much further up the overall table.

guardian1

guardian2

Our value added score has also decreased, even though the number of good degrees awarded has risen, this has been accompanied with a slight increase in entry qualifications. This needs further investigation at subject level, and by considering other universities – for instance Oxford have a value added score of 7.3, and their entry tariff is considerably higher than us at 583. It would suggest that the value added score is weighted much more towards the number of good degrees, rather than the input tariff.

So far, Executive have been supplied with a quick analysis of how the university overall has performed, and how we have performed in comparison to others.

guardian3

In the next couple of weeks, I will be working with Heads of Schools, Deans and Associate Deans to look in more detail at the subject tables, and to compare them with our own portfolio performance measures, Communications for all university staff will follow.

The most important thing to remember is that league tables aren’t “something that is done to us”. They are just a reflection of our performance. If we want to appear higher in the tables, then once we have dealt with any issues around data submission, then we need to address how do we improve our performance in student satisfaction, in attracting good students, and in ensuring that those students achieve the best that they can which will enable them to get suitable graduate employment.

 

More on the Complete University Guide

Looking at the data for all universities in the 2014 Complete University Guide, I couldn’t resist plotting a few graphs.

Firstly the relationship between entry qualifications and the number of students awarded good honours degrees.

degree-quals

 

Maybe this isn’t too much of a surprise – it appears to show that students who arrive at university with better qualifications are more likely to get a good degree.

It does however also suggest that those universities who recruit students with lower qualifications are not adding as much value to those students as they might like to claim.

Secondly, we can look at the relationship between graduate prospects and entry qualifications.

job-quals

 

again, not surprisingly, this seems to show a link between entry qualifications, and gaining a graduate job. We are all aware of the tales of employers who are not interested now in anyone with less than a 2(i), so if students are not able to get to this level of qualification, based on their entry into HE, they are going to struggle to get graduate jobs.

Finally, there is the relationship between research score and overall league table score. It’s reasonably strongly weighted in this and other tables.

research

 

 

So the message would seem to be – to be successful in a league table such as this:

  • increase the number of good degrees awarded
  • recruit better qualified students who are more likely to get good degrees
  • with better degrees, more graduates will get graduate entry jobs
  • increase research assessment scores

In terms of how to do this, there are not likely to be simple answers, however here’s a few questions:

  • how could a learning and teaching strategy be used to enable a step change in the development of academic skills to improve degree outcomes?
  • what instruments and data can be used to identify which subjects, awards or even modules are not allowing their students to reach their full potential?
  • are particular groups of students less likely to be successful – and what interventions could be developed that would benefit all students?
  • how could a university help to develop students’ social capital to improve graduate prospects?
  • how do teaching led universities  ensure they have the right kind of research, that can provide a boost in league tables  but which can support their core mission?

 

 

 

Two items of league table news

We are moving inexorably towards the start of the league table season, and much hand wringing is to be expected if table positions do not reflect what universities believe about themselves.

Firstly the THE World University Rankings for Asia was published last week. Staffordshire University has had a long running partnership with City University Hong Kong, through their School of Continuing and Professional Education., so it is interesting to see that City U makes it into the top 20 of universities in Asia.

Secondly, an article I originally missed in the Guardian, reports on the relationship between league table position and student recruitment.

“Prospective students are increasingly influenced by university league tables when deciding where to study, according to research that found rises and falls within league standings provoking sharp changes in numbers of applications.

The research by economists at Royal Holloway, University of London, found that individual departments moving up a subject-level league table experienced a rise in applications of almost 5%, with the increase most pronounced among overseas applicants.

They also found that the influence of league table standings has increased since the introduction of tuition fees, suggesting that students are now more aware of the reputation and relative standings of university departments.”

So far, so much like announcing the Pope’s Catholicism, however the study found that the impact was not the same for all universities:

“the boost in applications only applied to university departments sitting towards the upper end of the tables, and especially those within the top 10%. The economists found students to be “more or less indifferent” to changes for departments in the bottom half of the tables.”

Which provides an interesting conundrum – the higher you get in the table, the more significant it might become.