Differences in Student Outcomes

Posted on March 28, 2016 by HAMLYN Mike

Successful outcomes for students are often used as a proxy for institutional quality, hence the use of good degree outcomes, or value added, in league tables. The forthcoming Teaching Excellence Framework will almost certainly look at student outcomes as a measure also. However, not all students succeed equally, and we know from our own work at StaffsUni of the gaps in attainment between different groups of students.

The recent Green Paper, as well as highlighting the possible future TEF, indicates the government’s desire to see an increase in numbers of students from the most disadvantaged backgrounds as well as looking to ensure that all students can achieve.

In the light of this, last Monday I attended a HEFCE conference in London “Addressing differences in student outcomes: Developing strategic responses”, which looked at the findings of research into differential outcomes from Kings College London, and was an opportunity to hear from others in the sector on how they are tackling these issues.

Sessions attended were: the introduction by Chris Millward, Director of Policy at HEFCE; a presentation by Anna Mountford Zimnars of KCL; a session by Sorana Vieru and Malia Bouattia of NUS, and finally a session by Philip Plowden, DVC of University of Derby.

These are my notes of the day. Copies of the presentations can be viewed here.

Chris Millward HEFCE Director of Policy

Chris Milward started by considering where the government is on this agenda, linking the Green paper, the Treasury plan and plans from BIS.

Government wants to see a more diverse range of backgrounds in HE, in terms of entry, success and outcomes. For instance: double the number of students from disadvantaged backgrounds by 2020; an increase in the number of BME students by 20% by 2020, and to the sector to address differences in outcomes.

This means more responsibility for universities together with strengthened guidance to OFFA and the potential role of the Office for Students. There is an anticipated stronger role in quality assurance processes through the impact of TEF and the future need to measure difference in outcomes based on data and metrics agreed by government. This will lead to more targeted funding together with more emphasis on meeting obligations.

The HEFCE analysis shows an attainment gap for BME students, based on A-level analysis and the more that you add in other factors, the bigger the gaps become.

In addition, when looking at POLAR3 domicile, then there are further unexplained HE outcomes.

When considering students with disability, then the data suggests that those students who received DSA support perform above average, while those without perform less well.

On postgraduate progression, there is currently an unexplained difference in outcomes based on POLAR3 quintiles.

When considering employment and looking at the 40 month survey rather than the 6 month DLHE, all POLAR3 quintiles have worse outcomes than quintile 5 and for professional employment in particular. There are worse outcomes for students with disability, irrespective of DSA and there are worse employment outcomes for all categories of BME students and particularly in professional employment. Finally on gender, men perform worse overall on employment, but better in professional employment.

The HEFCE approaches to working on closing the gaps in outcomes include:

National outreach programme
Funding for disabled
Supporting successful outcomes
Catalyst fund

ANNA MOUNTFORD ZIMNARS – KCL

Dr Zimnars presented the outcomes of major piece of research into differential outcomes, which is available here.

“Access without success is no opportunity”

The research considered three questions:

What is the pattern- empirical?
How do we explain it – causal model?
How do we change it effectively- policy and empirical?

The question was asked – “Do we need causality- if intervention works, does the causal model matter?”

Explained pattern of differential attainment using model that looked through a lens of macro/meso/micro levels and at experiences of preHE, HE and postHE.

4 explanatory dimensions were proposed:

Curricula and learning
Relationships -sense of belonging probably the most important factor
Cultural, social and economic capital
Psychosocial and identity factors

From the research, which involved asking questions of a large number of institutions, the level of awareness of the issue differed across institutions, although this may be changing now, possibly due to the proposals in TEF.

In terms of those institutions that tackled the differential outcomes issues the most successfully:

Whole institution effect is most successful
Need students academics and prof services working together
Bottom up approaches with strategic support
Universal and targeted interventions

Effective interventions were seen to be:

Improvements to T&L
Inclusive learning and curricula
Deconstructing assessment
Meaningful interactions
Role models and mentoring
Engagement with institution
Generally few evaluations especially a lack of long term evaluations

Ended with 5 groups of recommendations

Evidence base
Raising awareness
Embedding agenda
Staff as change agents
Students as change agents

Sorana Vieru and Malia Bouattia NUS

This presentation started from a previous NUS report, Race for Equality, and went on to look at a new NUS campaign on liberating the curriculum.

From previous NUS work, 42% of students said that the curriculum did not reflect their experiences particularly in history and philosophy. As well as looking at students as being in one particular demographic group, it was important to look at intersections between groups.

Work from NUS highlighted:

23% of black students described learning environment as cliquey
Disabled students more dissatisfied in NSS
10% of trans students not willing to speak up in class
Black students report lower levels of satisfaction on NSS on assessment and feedback

There was a focus on liberation-equality-diversity and the launch of a new campaign – “Liberate my Degree”. An online hub has been provided with resources for officers and reps with training resources to allow them to engage in debate in their institutions and to support becoming co-creators of curriculum.

Getting there – Helen Hathaway Philip Plowden

Speakers from University of Derby showed the pragmatic steps they have taken to challenge the gap in attainment between white and BME students.

In terms of background, the University has 28000 students, most of whom were state school sector. 20% of these self-identified as BME. The attainment gap was 24.6% in 2009-10. The impact of the work so far is the gap has closed to 12.4% in 14-15, although there was an increase in attainment across all areas this is a moving target.

Important thing is that there is no one single answer, so there was a need to stop looking and focus on the myriad interventions and see what impact they have.

No magic bullet
Post racial inclusive approach
Suite of different strategies needed

Four main areas of interventions are used: Relationships, academic processes, psychological processes, and social capital.

The project at Derby explored data (down to module level) and relied on the regular Programme health checks which used a digest of metrics including attainment by ethnicity. In these, the DVC meets with programme leads to engage with course teams at chalk face. Areas covered include: outcomes, finances reliance on clearing, and staff numbers. In particular the programme health checks looked at “spiky” degree profiles- looking at individual modules and gaps, not with an intention to play a blame game but to ask what is going right and ask others to consider that.

To support interventions, Derby developed PReSS- practical recipes for student success whch contains evaluations and case studies and can be used from: Http://uodpress.wordpress.com

The key lessons learned were:

No simple solution. Paralysis by analysis. Just have to crack on and do what works.
Learn from others
Post racial inclusive approach. Difficult to reconcile this with some of the morning’s talk. Is this unduly dismissive of liberation approaches
Importance of communication -degree of profile. But once in the mainstream it might get lost.
Need consistent way to measure attainment gap.
Important to evaluate interventions.

Points from Discussions

A lively discussion followed, and the following are just snippets of some of the topics – in some cases these reflect discussion we have had in our own institution, but I add them in almost as provocations for further debate.

Is there a threat to academic staff when we discuss this BME and other attainment gaps? A danger of appearing accusatory?
Why are there difference between subjects such as business and nursing – do cohorts have an impact? Why do the subjects with the smallest attainment gaps want to engage in the debate the most?
How do we check who uses the resources to support inclusive learning, and should we check?
How do you liberate the curriculum and how do we re-educate staff to draw on a wider range of ideas, since they are a product of their own subject and environment?
What about the Attainment gap for students who live at home where home life and working gets in the way of study?

Conclusions

In all, a thought provoking day. A lot of emphasis, as always on the BME attainment gap, but also more opportunity to explore attainment more generally and to recognise how this agenda will become increasingly important post-TEF.

In terms of what we could do next, then as we develop better internal metrics of modules and courses, we can start to see how we can use this information to understand better the outcomes that our students achieve. Linking this to revisions in the way in which we review our courses, both from a quality assurance and enhancement perspective, as well as a more data-centric health check would provide the opportunity to have the right discussions, to ensure that we maximise the opportunities for our students to be successful.

Non Continuation Rates

Posted on March 27, 2016 by HAMLYN Mike

Last week, HESA published their latest data on student continuation rates.. An important set of figures for a number of reasons: non-continuation is something that directly affects the finance of universities; non-continuation is potentially a failure for the individual as well as the institution, and finally this data is used in some league tables.

A concern is that overall, the non continuation rate has risen across the sector (and indeed for us at Staffordshire University), with the national figure rising from 5.7% to 6.0% of students who entered in 2013-14 not progressing to the second year.The headline statistics are

6.0% of UK domiciled, young, full-time, first degree entrants in 2013/14 did not continue in higher education in 2014/15.
10.2% of UK domiciled, full-time, first degree starters in 2013/14 were projected to leave higher education without gaining a qualification

Usefully, HESA provides breakdowns of the data by both age of students as well as POLAR3 low participation indicator. This doesn’t necessarily provide any greater detail than that already held by any individual institution, but it does allow for comparisons to be made against comparators.

Looking at the data for Staffordshire University we can see that :

	Percentage no longer in HE (%)	Benchmark (%)
young entrants	12.2	10.1
mature entrants	14.1	13.8
all entrants	12.8	11.4

young entrants from low participation neighbourhoods	15.7	11.2
young from all other neighbourghoods	11.2	9.7

So, no surprises there, but it does add to weight to the argument that we should revise the way in which we look at the necessary interventions to support retention. If, as is evidenced here, there are groups of students who are more likely to withdraw than others, then a “one size fits all” approach to student retention will not deliver all the necessary outcomes.

In addition, HESA provide data on non continuation rates based on subject studied as well as entry tariff and types of qualifications. The rates compared to entry are summarised as:

Entry qualifications	All subjects

01 A level/VCE/Advanced Higher grades AAAA or Scottish Highers grades AAAAAA	1.4%
02 A level/VCE/Advanced Higher grades at least AAA or Scottish Highers grades at least AAAAA	1.8%
03 A level/VCE/Advanced Higher grades at least AAB or Scottish Highers grades at least AAAAB or AAAAC or AAABB	2.5%
04 A level/VCE/Advanced Higher grades at least AAC	3.1%
05 A level/VCE/Advanced Higher grades at least ABB or Scottish Highers grades at least AAABC or AAACC or AABBB or AABBC	3.1%
06 A level/VCE/Advanced Higher grades at least ABC or BBB or Scottish Highers grades at least AABCC or ABBBC or ABBBCC or ABBBB or BBBBB	3.9%
07 A level/VCE/Advanced Higher grades at least ACC or BBC or Scottish Highers grades at least AACCC or ABCCC or BBBBC or BBBCC	3.9%
08 A level/VCE/Advanced Higher grades at least BCC or CCC or Scottish Highers grades at least ACCCC or BBCCC or BCCCC or CCCCC	4.2%
09 Tariff points > 290	4.8%
10 Tariff points > 260	5.3%
11 Tariff points > 230	6.6%
12 Tariff points > 200	7.4%
13 Tariff points > 160	9.2%
14 Tariff points > 100	11.3%
15 Tariff points > 0	12.9%
17 Level 3 and A level equivalent qualifications with unknown points	13.9%
19 International Baccalaureate	3.4%
20 HE level foundation course	6.1%
21 Access course	11.1%
22 BTEC	11.5%
23 Higher education qualification – Postgraduate	7.1%
24 Higher education qualification – First degree	7.6%
25 Higher education qualification – Other undergraduate	8.1%
26 No previous qualification	24.1%
27 Other qualifications not given elsewhere	17.0%
28 Unknown qualification	32.6%

All qualifications	6.0%

Or looking at this graphically:

Important lessons from this data? As A level tariff points decrease, then the likelihood of non-continuation increases. Also, for institutions or courses that recruit significant numbers of students with BTEC qualifications, then higher withdrawal rates might be expected

Putting these factors together: age, POLAR3 neighbourhood, subject and entry grades, we can use better data analytics, linked to market segmentation and enhanced personal tutoring, to identify how to provide right support to all students, but in a way that is tailored to their needs and expectations. The key part of this will not be the identification of possible at risk students – the more difficult work will be in deciding what are the interventions needed to support an increasingly diverse range of students, and how to deliver this.

Ultimately, we want all of our students to succeed, and if we have decided that these are the people that we want to educate, then we have to provide the best opportunities for that success.

HESA Data Release

Posted on January 17, 2016 by HAMLYN Mike

HESA have just published their statistical first release for student enrolments and qualifications obtained at Higher Education providers in the United Kingdom 2014/15.

This is always a useful summary, to see the size of the HE “market”, and whch subjects appear to be growing or in decline, data which of course can be cross-referenced to UCAS data releases to to see how trends in applications map to trends in enrolments.

The headline data shows nothing new – the total number of students engaged in HE study dropped by 2%, largely due to the 6% drop on part time enrolments. Part time still continues to be a problematic area for the sector.

hesa1415_chart_1

In terms of subjects, we can see how individual subject areas are growing or in decline, which should influence the way in which institutions might want to proactively manage their portfolio.

The latest information shows that the areas of growth for undergraduate study are: biological science, computer science, subjects related to agriculture, engineering and technology, with the biggest gain in creative arts and design. On the other hand, there has been a sector wide drop in enrolments at undergraduate level again in languages, but also in business, law, history and philosophy, and education.

hesa1415_chart_5

On attainment, and an area of interest in light of comments on possible grade inflation in the recent discussions around the Green Paper, HESA note that “of those gaining a classified first degree, the proportion who obtained a first or upper second has shown a steady increase from 64% in 2010/11 to 72% in 2014/15. In 2014/15, 22% gained a first class degree compared to 15% in 2010/11.”. This steady rise will be reflected in league tables of course, but importantly for my own institution, our good degree rate has risen (not to the sector average), but to a defensible level.

Looking at data n where students come from, we can see that the UK is still a desirable location for HE study. Considering English HEIs only, the data shows:

hesa14-15 domicile

Not surprisingly we see that China remains the biggest provider of students to English HEIs, and continuing drop in students from India, Pakistan and Saudi Arabia, while there has been a big rise in students from Hong Kong.

As always the HESA data release provides excellent background information for anyone wanting an understanding of the shape of the UK HE sector, and where the trends are in types of students, their level and mode of study, their domicile, their outcomes and the attractiveness of the various subject groups.

Differences in Degree Outcomes

Posted on September 20, 2015 by HAMLYN Mike

New from HEFCE this week, a report on “Differences in Degree Outcomes:the Effect of Subject and Student Characteristics“, which looks at the outcomes of students who graduated in 2013-14. Some of this data I have previously reported when looking at HESA data on the impact of ethnicity on degree outcomes for the previous year.

The results of the HEFCE survey are not startling – they almost reinforce things that we already know in terms of what factors have an impact on achievement: the challenge now is to learn how to address each of these, and with the recent comments by the new universities minister on widening participation, and our own commitment to supporting a diverse population of students then awareness of these trends and how we then tackle them will be crucial for success of individuals and of the institution.

HEFCE considered the following variables when looking at the differences in outcomes:

age
disability status
ethnicity
The Participation of Local Areas measure (important for high WP populations)
sex
subject of study
prior attainment (in terms of qualifications held on entry to higher education)
previous school type
institution attended

The interesting part of the analysis is not the differences in outcomes that can be seen, but how much these differences can or cannot be explained by the influence of other factors.

Subject

Certain subjects are more likely to award 1sts/2(i)s, and the table below represents those subject we offer at Staffordshire – it will be interesting to compare our recent results with those for the sector by subject.

Subject	% first or upper second	% first
Subjects allied to medicine	69%	24%
Biological sciences	70%	18%
Physical sciences	73%	25%
Mathematical sciences	73%	35%
Computer science	66%	28%
Engineering and technology	74%	30%
Social studies	73%	16%
Law	69%	12%
Business and administrative studies	71%	21%
Mass communication and documentation	75%	15%
Historical and philosophical studies	82%	19%
Creative arts and design	72%	21%
Education	68%	18%
Combined	60%	16%

I always thought it was apocryphal that law didn’t award firsts – across the sector it would appear to be true!

Entry Tariff

On entry tariff, there is a clear relationship – higher entry leads to higher numbers of good degrees, which can also be seen when looking at league table data. This is one of the reasons that the Guardian league table uses a “value added” measure which seeks to adjust for entry tariff..

Mode of Study

In general, part time students have worse outcomes compared with full time. Even adjusting for variations on entry tariff, part time students have worse outcomes than full time.

Age

The raw data shows that young students are 11 percentage points more likely to gain a good degree compared with mature entrants.

Gender

Across all entry tariffs, women are more likely to gain good degrees than men.

Disability

Graduates with a disability are slightly less likely to gain a good degree than those without a declared disability.

Ethnicity

This is the area with the biggest gap. 76% of white students gain a good degree, compared to 60% of black and minority ethnic students.

Even allowing for other factors, the unexplained gap is still equivalent to 15%.

Previous School

In most cases students from state schools outperform those from independent schools.

Neighbourhood HE Participation

Students coming from neighbourhoods with the highest rates of HE participation also gain the highest numbers of good degrees.

Implications

The recent speech by Jo Johnson referred to the importance of universities in driving social mobility and the sector’s work in widening participation.

This data provides further information that could be used to justify the costs of supporting WP in universities, and for focusing on trying to close gaps in attainment.

Much focus is given to looking at the data provided by UCAS but to understand how well the sector and individual universities are performing in terms of closing these gaps, then much fuller datasets need to be considered, taking into account retention and progression and ultimately employment – even if all our students gain the degrees they deserve, but still fail to progress into appropriate graduate roles, then social mobility isn’t realisable for everyone.

As we move into a potential quality regime that could be metrics based, together with a Teaching Excellence Framework, which will certainly use a variety of metrics (possibly including learning gain), then there will be plenty of work to be done in generating data and analysing it..

However, the focus also has to go beyond analysing data. How can we use it to understand our students both as individuals and as cohorts? How can we use data to support our staff better in teaching and assessing their students? Finally, how can we learn to change practices and behaviours based on evidence?

Do the numbers matter?

Posted on August 24, 2015 by HAMLYN Mike

We are now at the point in the year where we start getting hold of course level metrics – from employability through DLHE, for student experience from NSS and on student performance in terms of retention and attainment through our own datasets.

Bringing these together means that we can create a snapshot of how “well” a course might have performed in the last years.

There have been a number of publications over the summer on the use of numbers and metrics, in particular the report “The Metric Tide” which reflects in the use of metrics to assess research excellence.

However this publication also contains chapters on management by metrics and on the culture of counting, and as someone who works extensively on looking at the performance of our portfolio of courses, as well as league tables, this was of interest.

“Across the higher education sector, quantitative data is now used far more widely as a management aid, reflecting developments in the private sector over recent decades. ……………….., most universities now plan resource allocation centrally, often drawing on the advice of dedicated intelligence and analysis units that gather information from departments and faculties. The use of such systems has helped universities to strengthen their reputation as responsible, well-managed institutions. The relatively robust financial position of the sector, and the continued trust placed in universities by public funders to manage their own affairs, is in part founded on such perceptions of sound financial governance.

The extent to which management systems in HEIs help or hinder institutional success is of course contested. On the positive side, such systems have helped to make decision making fairer and more transparent, and allowed institutions to tackle genuine cases of underperformance. At the same time, many within academia resist moves towards greater quantification of performance management on the grounds that these will erode academic freedoms and the traditional values of universities. There is of course a proper place for competition in academic life, but there are also growing concerns about an expansion in the number and reach of managers, and the distortions that can be created by systems of institutionalized audit.”

What is important then is how we deal with data. A list of numbers alone does not create useful management information. Indeed even a collation or aggregation of all the data (similar to a league table approach) still is only one part of the picture.

What data or information such as this does provide us with, are some insights into how different parts of the university are faring, or how our different groups of students see us.

The useful work starts when we realise how to use the numbers – this is where we now have those conversations with course teams to find out why a metric is particularly high or low. Is there some really great practice that can be shared with other people? Is there a reason for a disappointing NSS score?

Only by going beyond the numbers and engaging with the course teams will we get the full insight into why the results are as they are.

This is not to say that everything can be explained away. The whole point of building up a metrics approach to assessing what we do is threefold:

To make sure all colleagues are aware of how measurable outcomes affect us reputationally and reflect the results and experience of actual students
To provide a consistent reliable management information to act as a trigger
To raise the data understanding capability of all groups of staff.

We should not be afraid of looking at metrics to judge a programme, but as well should become better at using that information to be able to understand exactly why we perform that way.

As well as looking at the raw data, we also need to look closely at what it is we are trying to achieve, and how this might influence how we set up benchmarks and targets. Some examples might be:

Benchmarking NSS results for subjects against the sector average for that subject. This shows how well we do in comparison with others rather than a comparison against an internal university average score (guess what – half our courses were above average)
Considering a calculation of value added instead of good degree outcomes. For a university with a significant intake of widening participation students, this might be a better reflection of “distance travelled” and show the results of our teaching. Any VA score should have to be different form that used in one of the league tables, which only considers 1sts and 2(i)s as a good outcome. For some students, a 2(ii) might be appropriate.

We should all be aware that using metrics to assess quality and performance is becoming increasingly important.

The current consultation from HEFCE on the future of quality assurance has a number of major themes, but two of these are around data and governance.

In the proposals are the suggestions that quality could be assured by a university identifying its own range of measures that indicate quality, and that governing bodies will be in a position to make judgements of success against these.

This could be an opportunity to create a set of metrics that really measure where we want our successes to be and that are actually aligned to the mission of the university., rather than the ones that might suit another university more readily.

Secondly, it does mean that governing bodies (and the people that brief them) will need to become more aware of data, its limitations and meanings.

Finally, and this is a concern – the proposed Teaching Excellence Framework will most likely be put in place very quickly, and will be metrics based. In the time available, this might only be based on metrics and measures that are already well known and used – NSS, DLHE, good degrees (not dissimilar to a league table so far). Since the ability to charge increased fees will depend on success in the TEF, then it does mean that despite in future possibly being able to identify what our measures of success will be, in the short term we cannot stop focussing on those key indicators.

Data, data everywhere

Posted on June 28, 2015 by HAMLYN Mike

A welcome publication this week from the British Academy “Count us In” identifies the importance of the ability to understand and interpret data in the 21st Century.

The ubiquity of statistics makes it vital that citizens, scientists and policy makers are fluent with numbers. Data analysis is revolutionising both how we see the world and how we interact with it.

This new report from the British Academy offers a vision of how the UK can rise to the potentially transformational challenge of becoming a data-literate nation.

Within this are two clear messages for a university. Firstly how we make sure that the graduates that we produce are able to work and function in a data literate society, and secondly how we as an organisaton become more data literate.

Universities have traditionally been influenced by the liberal arts – and I will always defend the importance of developing high level critical thinking skills through a liberal education.. Howver, this in turn has influenced how we might define what is to be a graduate, with a focus on communication, reflection and team working. However, few universities have developed their definition of graduate skills or more recently graduate attributes to explicitly explain how a graduate will be numerate, be able to handle data and be able to make decisions based on proper analysis. More surprising, is that this flies in the face of the skills that we know that employers value. We would have to ask why we have shied away from putting quantitative skills front and centre.

The report from the British Academy envisions:

a generation of citizens, consumers, students and workers as comfortable with numbers as they are with words, confidently engaging with data in a future driven forward by technological development and a drive for international competitiveness.

and in doing so recognises the need for cultural change at all levels of the education system.

For UK universities, the key messages in the report are:

the need for universities to send signals to school on the importance of quantitative skills
the need for it to be normal for science,social science and humanities students
to have developed significant quantitative skills in school, so that universities can then strengthen their entry requirements.
the worry that we dilute the curriculum to reflect the current poor level of quantitative skills of students
the bigger worry that the changes in course design may reflect weaknesses in the quantitative and data skills of university teaching staff
students graduate with little confidence in these skills, which have a negative effect on the businesses they subsequently work for

In terms of moving forward, the proposals from the report include:

universities should review and if necessary redesign the content of social science and humanities degree programmes
universities need to signal with more clarity what level of quantitative skills is necessary for each course
an increasing need for collaboration between universities and employers to work with the data now collected and generated by the private sector

As well as the need to develop courses that develop quantitative skills, a university must also be aware of their own workforce’s skills. All staff in university might reasonably be expected to be able to handle data to make decisions – for instance through measuring student engagement as a personal tutor, optimsing a timetabling system, predicting recruitment numbers and workforce planning, benchmarking organisational performance through extenral data sets and league tables. The list goes on.

From the BA report, many more people in the workplace need to be able to handle data fluently.However:

a substantial body of case study research suggests that many employees fail to understand fully the quantitative techniques they are using, and lack the ability to recognise obvious errors in their work.

and

The almost universal investment in technology by private, public and voluntary sector institutions does not negate the need for numerical understanding. Rather, it adds
to it, as people require skills of investigation and interpretation. Nor are quantitative skills deficits confined to less senior employees: it has been estimated that as many as 58 per cent of people in “higher managerial and professional occupations” do not have numeracy skills at GCSE A*–C and above

All of which is worrying, as the report clealry identifies the economic benefits to organisations of being able to use data well.

To improve the situation for companies, the report proposes internal staff development, and engagement by businesses with training providers including FE and HE and taking advantage of apprenticeships.

I hope that this report helps spark more conversation -maybe even a strategic discussion at a committee somewhere – on the need to improve numeracy, quantitative skills and data analysis.

For a university there are two key main areas to debate:

Firstly, how do we explicitly improve the quantitative skills of all students, and how do we show this to potential employers that this is the factor that differentiates our graduates?

Secondly, how do we raise the data handling skills of all of our staff – teaching and professional support – to be able to teach and use data in the most effective way for the organisation?

To make a small step forward on this, tomorrow I will be presenting at our “Leading Academics” course on how to use data with the following outcomes:

To recognise the importance of using performance data
To identify which parts of the performance data set might be a priority for action within own subject area
To understand the benefits but also limitations of metrics based approaches

It’s a start.

Is UK HE lagging behind the global race?

Posted on June 20, 2015 by HAMLYN Mike

This year’s annual survey of Vice-chancellors and report by PA Consulting has just been published – and for colleagues at Staffordshire, it’s always good to read the work of Mike Boxall, who presented his ideas on Oligarchs, Innovators and Zombies at our Leadership Conference last year.

This year’s report, “Lagging behind: are UK universities falling behind in the global innovation race” takes a different approach – and looks at innovation in HE, and which developments in teaching and learning are seen as important.

So, our VCs think that the UK is lagging behind in every major area of innovation, and propose the following as the reasons for this:

deep seated conservatism of university cultures
constraints of inflexible organisational structures
fragmented and tentative nature of change initiatives
perceived lack of incentives for innovation
improved confidence in resilience of sector
widely held views that current models of HE provision and participation will remain the same for years to come

Even before reading the conclusions of the report – this seems worrying. Senior university leaders think that UK HE is lagging behind global competitors, in an increasingly globalised market, and propose a series of reasons that could explain this. Maybe I misread the memo, but remind me, who is able to lead changes to culture, organisational structures and change initiatives?

The report identifies the paradox between a residing belief that the main university experience in 15 years time will still be the full time 3 year undergraduate degree. Arranged against this are the promoters of “disruption”, led by Clay Christenson and his various acolytes (Sir Michael Barber, Sebastien Thrun et al who believe that “education is broken”).

Somewhere between these two extremes however is where change will actually happen. PA identify 7 themes that they believe will transform HE globally (for more on technology changes, its worth looking at the work of Educause and the NMC Horizon Report).

From the survey, the three themes identified as essential to survival were :

technology to transform learning
integration with working practice
student data analytics

Essential to maintaining competitiveness were;

student data analytics
integration with working practice
curriculum reforms

Technology to transform learning is a given. All of our students arrive at the university with a high level of digital capability.. First year 18 year olds do not remember a life without fast internet, with Google and Wikipedia on hand to provide information. Other students who come from employers will already be used to technology as a key part of their lives. We need to get better at recognising and uderstanding the digital skills of our students, how they differ from our own, and which digital capabilities we need to develop in both staff and students. Walking around with an iPad does not make you a digital native or resident, but realising how you can use it to create, curate and communicate learning is a start.

Data analytics is seem to be crucial for both survival as well as competiveness, which is interesting since use of student data analytics is still limited within the sector, with 2/3 of VCs surveyed saying they had made little or no progress in this area. So far we might have developed plenty of data on students who apply to us through UCAS and universities have developed plenty of market intelligence to drive recruitment, but analytics will mean more information on the performance, attendance and engagement of students. This nascent “big data” approach will potentially provide really useful information to all levels of staff in the organisation, and there are plenty of companies wanting to sell these technologies to the HE sector. Time to beware the snake oil salesmen.

Working with employers and accreditation of work experience are approaches that will be readily recognised in the sector by newer universities, although maybe more of a challenge to understand by the more established residents in the marketplace. An increase in working like this will inevitably mean a greater shift from the traditional three year degree though – which does conflict with the view that this will remain the dominant form of HE.

PA conclude their report with:

The challenge for UK universities is not a failure to recognise the needs and opportunities for innovation, nor is it a lack of evidence for successful innovations elsewhere. Rather it stems from the profound difficulty of innovating in inherently conservative organisations that are still doing reasonably well from their old ways of working. Most universities can point to examples of innovative initiatives in their curriculum, pedagogies and student experiences, but these are almost all localised within the organisation and tentative in their scale and commitment. Meanwhile the core ‘business-as-usual’ of most institutions remains much
as it has been for many years, with diminishing relevance and value to changing student needs and expectations.

In summary, for me this report presents a distillation of key trends, but also a range of frustrations – if we can recognise what the limits are to innovation, then we need to find ways of fixing them and removing the barriers to development.

Technology is going to be key to future developments, in learning, in analytics and in measuring the performance of an organisation, which reinforces the need for an increase in digital capability at all levels in a university organisation, as well as having a clear technology vision and strand to any operating plan.

“Good” degrees – but not for everyone

Posted on February 22, 2015 by mikehamlyn

In a recent post I looked at the latest HESA data on the numbers of 1sts and 2(i)s awarded, noting the continued rise, and how these figures feed into the various league tables.

I suggested then that the HEIDI data could be used to see how students from different groups perform – in fact this is how the Equality Challenge Unit annual statistical reports are compiled.

Having looked at the information from the last two years, then we can see the attainment gap for BME students for 2012-13:

	2012/13 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) White Classification of first degree 1st and 2(i)s	2012/13 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Black Classification of first degree 1st and 2(i)s	2012/13 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Asian Classification of first degree 1st and 2(i)s	2012/13 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Other (including mixed) Classification of first degree 1st and 2(i)s	2012/13 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Not Known Classification of first degree 1st and 2(i)s
Sector Average	69%	45%	55%	62%	44%
Gap		24%	14%	7%	25%

In 2013-14 this changes to:

	2013/14 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) White % 1sts and 2(1)s	2013/14 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Black Classification of first degree % 1sts and 2(1)s	2013/14 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Asian Classification of first degree %1sts and 2(1)s	2013/14 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Other (including mixed) Classification of first degree % 1sts and 2(1)s	2013/14 HE student qualifiers Full-person equivalent Ethnicity (detailed 6 way) Not known Classification of first degree %1sts and 2(1)s
Sector Average	71%	48%	58%	65%	45%
Gap		23%	13%	5%	25%

So we can see that the attainment gap across the sector is beginning to close, but its still a work in progress. The data I’ve used to create these summary results does provide results for each institution, however I won’t be publishing that here, as all universities are tackling these matters in their own way, depending on their particular subject mix and student population.

At Staffordshire we’ll be doing some focused work in two particular schools (both of which I am currently seconded to), as these are our schools with the most diverse undergraduate populations. Conversations with our staff have already started to identify differing levels of engagement and attendance, and we are now looking at many of the topics raised by Winston Morgan, in his talk here last year, for instance the use of appropriate examples in teaching materials, the composition of the teaching team and the need to provide positive role models in an institution where the mix of people in power may not fully reflect the student body.

As well as considering ethnicity, we also need to look at how disability can affect student attainment. In general, disability has less of an impact on degree classification than ethnicity, however BME student with a disability are less likely again to gain a good degree, as shown in this data from the most recent ECU “Equality in higher education: statistical report 2013” :

None of this is going to be easy, but if we want to ensure success for all students then it’s an issue we need to tackle head on.

“Good” Degrees

Posted on February 15, 2015 by mikehamlyn

We all know that gaining a good degree is important, perhaps more so now than ever. The increasingly consumerist approach by students might be enshrined in “what do I need to do to get a 2(i)?”, but in many cases this is also accompanied by a commitment to work that was perhaps less of a focus when I first studied. That might be also be attributable to the changing perceptions that students have of their higher education – seeing it as a transaction in which they engage to gain clearly defined outcomes, rather than the wider exploration that HE might have been considered to have been in some non-existent golden era.

A good degree is understood to be a benefit to the individual – it’s likely to help open doors in getting that first graduate job. It’s also beneficial for institutions for their students to be successful in this way: all university league tables include “good degrees” or some variant thereof in their analysis, and so the university that awards high numbers of good degrees can expect to reap the rewards in league table position. Of course there is also virtuous circle effect here – universities that are at the top of the tables may be the most selective, and able to recruit the students with the highest entry tariff scores in the anticipation that they will thrive. Other institutions will argue that they provide a greater amount of value added to students with lower entry grades.

In January, HESA published its first data release, which showed the range of degree classifications as follows:

72% of first degrees undertaken through full-time study in 2013/14 achieved first or upper second classifications compared to 54% of those undertaken through part-time study.

Now that more detailed data has become available through Hedi, then we can look to see how the different institutions perform on this measure – and whose outputs have changed significantly.

So here are the top 10 universities for awarding good degrees in 2013-14:

Institution	2013 % 1sts and 2(1)s	2014 % 1sts and 2(1)s	difference
The University of Oxford	92%	92%	0%
Conservatoire for Dance and Drama	91%	91%	0%
Guildhall School of Music and Drama	87%	91%	4%
Central School of Speech and Drama	88%	88%	0%
The University of St Andrews	88%	88%	0%
The University of Cambridge	87%	88%	1%
University College London	87%	88%	1%
Royal Academy of Music	77%	88%	11%
Imperial College of Science, Technology and Medicine	88%	87%	-1%
University of Durham	85%	87%	2%

And at the other end of the results….

Institution	2013 % 1sts and 2(1)s	2014 % 1sts and 2(1)s	difference
London Metropolitan University	51%	55%	4%
University of Bedfordshire	48%	55%	7%
The University of East London	54%	54%	0%
Glynd?r University	54%	54%	0%
University College Birmingham	46%	54%	8%
University Campus Suffolk	56%	53%	-3%
University of Wales Trinity Saint David	49%	51%	2%
SRUC	44%	51%	7%
The University of Buckingham	43%	51%	8%
The University of Sunderland	54%	50%	-4%

For those of us who have an interest in league tables, then the interesting thing to look at will be those universities which have seen significant changes in the percentages of good degrees that they award. Hence we might look to see some league table gains (ceteris paribus) for the following:

Institution	2013 % 1sts and 2(1)s	2014 % 1sts and 2(1)s	difference
Leeds Trinity University	56%	69%	13%
Royal Agricultural University	51%	63%	12%
Royal Academy of Music	77%	88%	11%
Bournemouth University	65%	76%	11%
Glasgow School of Art	59%	69%	10%
The University of Wolverhampton	50%	59%	9%

noting that Wolverhampton doesn’t engage in league tables.

The biggest drops are for:

Institution	2013 % 1sts and 2(1)s	2014 % 1sts and 2(1)s	difference
University Campus Suffolk	56%	53%	-3%
Writtle College	52%	49%	-3%
Heythrop College	83%	79%	-4%
Royal Conservatoire of Scotland	79%	75%	-4%
The University of Sunderland	54%	50%	-4%
The Royal Veterinary College	75%	66%	-9%
University of the Highlands and Islands	71%	58%	-13%

As well as looking at the percentages of good degrees, with a little bit of Heidi magic we can look to see how various student characteristics have an impact on outcomes. A particular interest of mine is attainment of students from a BME background, and in considering how any attainment gap can be reduced. This will form the subject of a later post.

Equality Unit Statistical Report 2014

Posted on November 23, 2014 by mikehamlyn

Last week, the Equality Challenge Unit published its annual statistical report which considers a range of data sets from HESA, relating to both staff and students. This provides a great insight into the diversity of all the people engaged in the UK higher education sector, but also provides data against which we can benchmark ourselves for activities such as Athena Swan or Race Equality Charter Mark.

A particular interest of mine is student success and attainment, so turning to the statistical reports on students we can see the following:

-The ethnicity degree attainment gap has decreased from a peak of 18.8% in 2005/06 to 16.1% in 2012/13, and is at its lowest since 2003/04. Nevertheless, the gap in attainment compared with UK-domiciled white first degree qualifiers remains considerable, particularly for UK-domiciled black: African first degree qualifiers (with a gap of 26.8%) and UK domiciled black: Caribbean first degree qualifiers (24.5%).
-The ethnicity degree attainment gap was larger among UK domiciled first degree qualifiers who studied non-SET subjects than among those who studied SET subjects.
-In every subject, a higher proportion of UK-domiciled white first degree qualifiers received a first/2:1 than UK-domiciled BME first degree qualifiers.

So there is still considerable work to be done, firstly to really understand the causes of the attainment gap, but much more importantly, to put interventions into place that will help to remove it. Some of the ideas at our Learning Teaching conference in the summer from Dr Winston Morgan of UEL are worth revisiting.

Any university that is trying to reduce the attainment gap has to be mindful of the classification of “BME”. This aggregation is not always helpful, and students from different ethnicity may have a range of different expectations and backgrounds that may affect their engagement and success. More useful is for an individual department in a university to gain a clear understanding of its own student body, their educational backgrounds etc, and then to review past performances on a more granular level, so that all involved in recruitment and teaching have a clearer idea of what the student population actually comprises.

When students with disabilities were considered, then in 2012-13 the percentage of students with declared disabilities gaining a first or 2(i) rose from the previous year.

In addition, the gap in attainment between students with or without a disability is much smaller than for the attainment gap seen for BME students:

These two sets of data reflect what we have seen previously at Staffordshire University – a small (and sometimes insignificant) attainment gap for students with disability,but a significant attainment gap between white and BME students.

All in all, the statistics from ECU provide some really useful background information for universities as they progress their equality and diversity agenda.

Mike Hamlyn

another blog about HE in the UK

Category Archives: Data