This is second part of the three part series:
The two important tertiary education problems you won’t hear about this election year
This series of articles looks at difficult issues, wicked problems that face the tertiary education system, areas where the system is clearly failing.
But really difficult issues don’t usually carry any currency in an election year. The lead governing party won’t want to answer questions about why a failing policy was allowed to fester. The lead opposition party won’t want to be facing searching questions on complex problems.
Election campaign debates are for hot issues – things like how good/bad the Te Pūkenga venture has been. Or like should we have a new medical school rather than expanding training in the existing medical schools. Or questions about higher education finances. Or even the upcoming review of the HE funding system. Not for complex evidence-based solutions that may take years to develop and resolve.
This series asks: what if the country took the opportunity to fix a part of the system that is clearly failing? How should we take a fresh look at a seemingly intractable problem?
In the first post of this series, I looked at the student allowances scheme and explained how the evidence showed major flaws in the design of the scheme. This article reviews another important aspect of our tertiary education landscape – programmes designed to help young people at risk of poor employment outcomes and explains why it’s failing, why it represents a difficult problem.
The third article in the series, due in a few days, will ask how a new government might take some steps towards a real, step-change improvement in the failing areas described in this post and in the first part of the series.
What’s the problem?
Many young people leave school facing the risk of long-term limited employment – spells of unemployment and benefit dependency, periods unemployed, periods of under-employment, periods on the minimum wage, low-skill work, and precarious employment and periods of enrolment in low-level tertiary education[1]. At the same time, New Zealand faces shortages of skilled workers as work becomes more sophisticated and requires more and different skills – core cognitive skills and, equally, socio-emotional skills[2].
Overwhelmingly, young people at risk of limited employment have no or low school achievement. New Zealand is one of seven countries that participate in the OECD’s PISA programme where the trend in performance in literacy, mathematics and science is declining[3]. That is an indicator that the number of people completing their compulsory education with low school achievement and low cognitive skills – and hence, at risk of long-term limited employment – is rising, rather than falling. This lead indicator suggests the problem is likely to get worse in the near future.
What we know from past experience is that when a recession hits, when employment gets tight, when unemployment climbs, it is the young and the unskilled who suffer most. The unskilled young people especially.
Look at how unemployment spiked during the global financial crisis (GFC) that hit in 2009.
Unemployment began to rise slowly among the young in 2007 as the first effects of the GFC began to hit and then soared – the unemployment rate among those aged 15-19 hit nearly 25% in 2012[4].
When the labour market is stronger, unemployment falls and employment increases, but the pool of unskilled and unqualified people will face only precarious employment, under-employment, low wage employment – and are more likely to be discarded.
Studies of young people’s disengagement from the labour market[5] cite evidence that inactivity and detachment from the labour market at an early age has long term scarring effects. It is likely to lead to unemployment, lower earnings, low-skill, precarious work. It is also associated with increased risk of becoming marginalised or involved in high-risk behaviour (for example, drug abuse or crime), housing problems or homelessness, poorer health and a lower quality of life. That leads to significant costs for the individual and for the society.
The problem of poor transitions to adulthood and to the workforce is among the most important facing our country. It is also a highly complex issue that cannot be addressed by ad hoc and disconnected interventions – we need a comprehensive and long-term approach.
Governments have faced this problem before…
Post-secondary education programmes to address the problems of youth at risk of long-term limited employment have been with us for decades. Access, Training Opportunities, Youth Training. Responsibility has moved back and forth between government agencies responsible for labour, welfare and education.
A MSD investigation of the outcomes of those who took a Training Opportunities programme course[6] checked whether and for how long completers received a welfare benefit in the six years after finishing the programme, finding that this group spent longer on a benefit than a matched comparison group. The analysis concluded “The increased time receiving Work and Income assistance shows participation in Training Opportunities does not increase the time participants spend in full time employment or tertiary study.”
A Ministry of Education analysis of the outcomes of the Youth Training (YT) programme[7] suggested that trainees who had had some employment experience before starting the programme were more likely to be in work two months after completion. As were those who used a series of consecutive spells on YT to accumulate unit standard credits. However, what really mattered for success in the labour market was labour market conditions rather than factors associated with the design of YT; controlling for labour market conditions, the effect of the programme on employment was slight.
Youth Guarantee, Youth Service
In 2008, the government began working on the SchoolsPlus initiative while the opposition came up with the idea of a programme to be called The Youth Guarantee. The two were similar in that there was an emphasis on addressing young people’s educational achievement, in the expectation that, if students could be given more skills and if those skills could be credentialed, then that would provide a better protection against the risk of unemployment.
Because National won the 2008 election, it was the Youth Guarantee that was implemented in 2010. This is a fees free programme for 15- to 24-year-olds who had left school with low or no qualifications. It aims to re-engage its participants in education in order to provide a pathway into further study, training and employment. The immediate target was educational achievement – NCEA at levels 1 and 2 and/or a Level 3 tertiary qualification. YG provides funding for providers to support learners’ travel costs. And providers now also are required to work with each learner to develop a “pathway plan” and they are funded to provide pastoral care to learners.
The Youth Training programme was either phased out (or incorporated into YG – take your pick) in 2012.
In 2009, YG looked like a great initiative, well-timed to create an option for the young as the GFC loomed.
In 2012, MSD introduced a companion programme, the Youth Service Ratonga Taiohi which offers support in gaining places in education, finding jobs, advice on financial assistance, accommodation support etc to young people at risk.
So, how has it worked out?
Youth Guarantee fees free programme
One of the most useful features of YG was that a comprehensive, regular evaluative monitoring programme was part of the design. Five monitoring reports were published, the most recent in 2018[8].
The monitoring looked at what happened to those who took the programme and compared that with young people who were similar but who did not participate. That comparison group was selected to match the participants in age, gender, ethnicity, educational achievement and social and parental background[9].
That comparison gives an idea of whether the outcomes – the educational achievement and labour market outcomes – were likely to be due to participation.
What we see is that YG succeeded in keeping people in education longer and those who took YG were more likely to get NCEA Level 2.
But employment outcomes were problematic.
The proportion in full employment two years after starting the programme was the same as the comparison group. And the YG participants were more likely to be NEET. And more likely to be receiving a benefit than the comparison group.
In other words, YG had some education benefits, but those benefits didn’t translate into employment. YG did not reduce the risk of long-term poor outcomes. Yet YG is one of the government’s main tools for supporting this vulnerable section of our youth population.
Youth Service Ratonga Taiohi
The Treasury published evaluations of the three strands[10] of YS in 2016[11], using an analytical approach similar to David Earle’s YG monitoring report discussed above.
The evaluations find that YS increases the retention of participants in education and improves their educational attainment. YS NEET leads to no improvement in the likelihood of employment. All strands of YS appear to raise, rather than lower, participants’ subsequent benefit receipt rates, (although the evaluators found evidence that recipients of the YS Young Parent Payment are somewhat more likely to move off a benefit after 24-30 months).
Meaning?
If I were being especially generous, I would rate this as mixed results. But it’s eight years since the latest monitoring report on YG came out with that depressing conclusion – YG doesn’t work as a means of helping young people into sustainable employment. YS is little better.
The protections put in place by governments over two or more decades haven’t resolved the problem. Significant numbers of young people are leaving school with low achievement and facing limited employment, low paid work, precarious employment, spells of unemployment. Yes, it’s a tough problem. Can’t we do better?
Let’s take a first look at what’s gone wrong
Intervening early, intervening late
Exploring the formation of skills in childhood, Nobel prize-winning economist James Heckman[12] suggests that ability gaps between individuals and across socioeconomic groups appear at an early age, for both cognitive and socio-emotional skills. They argue that interventions should occur early in life[13] because, as children progress through school, gaps become entrenched or grow; after year 2, schooling tends to play only a minor role in reducing gaps[14].
Children’s skills are highly correlated to family background factors like parental education and maternal ability; when those two factors are statistically controlled for, gaps disappear or are reduced[15].
Heckman’s group argues that the later an intervention is given to a disadvantaged child, the less effective it is likely to be. And later interventions are more likely to be more costly[16].
In other words, the problem we are addressing with these programmes is fated to be really challenging and really expensive. The people offering YG and YS face the most difficult task in the system.
It’s a question of stocks and flows. Fix a problem at source and it stands a chance of staying fixed, more or less, as the person progresses through the education system. Let it fester and it will fester. It is the job of the tertiary system to deal with the stock, the festered sore. It is government’s job to look at both.
Look at best practice …
Having found that YG wasn’t hitting the mark, the Ministry of Education commissioned a best practice review. This review, by Mandy McGirr and David Earle, looks at the evidence of what works in programmes like YG that are pitched at those at risk at the point of transition out of school[17]. They find that effective skills training programmes share a range of characteristics, including:
- including a work experience or on job training component
- combining with job seeking assistance
- being tightly targeted to the needs of a certain group
- being aligned to specific skill shortages for identified industries or locations
- including supports like personal coaching, mentoring and case management and tailoring of individual plans for learners.
We see that pastoral support and individual plans have been added to the requirements for YG providers, but for the rest ….
The findings of the McGirr Earle review should inform both programme practice and, especially, programme design.
A really useful contribution to the discussion
Back in 2020, the new Public Sector Act 2020 placed an obligation on agencies to develop independent strategic, long-term insights briefings on matters of importance, every three years (or more frequently).
MBIE, the Ministries of Education and Social Development and the Ministry for Women have joined forces to do a long-term insights briefing that looks at the topics covered in this article[18]. It’s an excellent read.
That paper will help shape Part 3 of this series, in which I lay out some actions the incoming minister – whoever that is – should take on this topic and on the vexed question of what to do with the failing student allowances system.
References
This comment drew on the following …
Carneiro P, Cunha F and Heckman J (2003) Interpreting the evidence of family influence on child development Research Gate
Cunha F and Heckman J (2007) The technology of skill formation NBER Working Paper No 12840 National Bureau of Economic Research
Dixon S and Crichton S (2016) Evaluation of the impact of Youth Service: NEET programme, Treasury Working Paper 16/08, The Treasury
Earle D (2018) Monitoring Youth Guarantee: Youth Guarantee Fees-Free places, Ministry of Education
Heckman J (2011) The economics of inequality: the value of early childhood education American Educator, Spring 2011
Kautz T, Heckman J, Diris R, ter Weel B and Borghans L (2015) Fostering and measuring skills: improving cognitive and non-cognitive skills to promote lifetime success OECD Publishing
McLeod K, Dixon S and Crichton S (2016) Evaluation of the impact of Youth Service: Youth Payment and Young Parent Payment, Treasury Working Paper 16/07, The Treasury
McGirr M and Earle D (2019) Not just about NEETs: a rapid review of the evidence on what works for youth at risk of limited employment Ministry of Education
NZ Productivity Commission (2020) Technological change and the future of work Productivity Commission
OECD (2019) PISA 2018 results: what students know and can do: Volume 1 OECD Publishing
Pacheco G and Dye J (2013) Estimating the cost of youth disengagement in New Zealand AUT
Mahoney P (2010) Youth Training: statistical profile 1999-2008 Ministry of Education
MBIE et al (2023) Preparing all young people for satisfying and rewarding working lives MBIE, MSD, Ministry of Education, Ministry for Women
Nedelkoska L and Quintini G (2018) Automation, skills use and training OECD Social, Employment and Migration Working Papers, No. 202, OECD
Samoilenko A and Carter K (2015) Economic outcomes of youth not in education, employment or training (NEET) Treasury Working Paper 15/01 The Treasury
Stolte O (2004) The measurement of Training Opportunities course outcomes: an effective policy tool? Social Policy Journal of New Zealand: Issue 21
Thomas S, Meissel K and McNaughton S (2019) What affects how often mothers read books to their preschoolers Ministry of Education
Endnotes
[1] McGirr M and Earle D (2019) Not just about NEETs: A rapid review of evidence on what works for youth at risk of limited employment Ministry of Education
[2] NZ Productivity Commission (2020) Technological change and the future of work Productivity Commission, Nedeloska and Quintini (2018) Automation, skills use and training OECD Social, Employment and Migration Working Papers, No. 202, OECD Publishing.
[3] OECD (2019) PISA 2018 results: what students know and can do: Volume 1 OECD Publishing. Note that the results of the latest (2022) PISA assessment are not yet available.
[4] Note that many of the unemployed 15- to 19-year-olds shown in Figure 1 were still in education, marking time while waiting, watching for work opportunities They weren’t all “on the couch”. But in every year between 2009 and 2012, more than 9% of 15- to 19-year-olds were not in employment or education or training (NEET), while among 20- to 24-year-olds, the NEET rate hovered around 18%.
[5] Pacheo and Dye (2013) and Samoilenko and Carter (2015) are two NZ studies of outcomes for disengaged youth. Note that both papers, like other research of that time. use “NEET” as a shorthand for detachment from the labour market and education systems. While the NEET rate is a useful statistical index for changes in the level of detachment, many of those NEET are not detached and many who are detached are not NEET – see McGirr and Earle (2019) which uses the term long-term limited employment. Their argument is, nonetheless, robust.
[6] CSRE (2011) Evidence on Training Opportunities and related training programmes, Centre for Social Research and Evaluation, Ministry of Social Development
[7] Mahoney P (2010) Youth Training: Statistical profile 1998-2008 Ministry of Education. Note that this analysis used as the indicator of a successful outcome the same measure that MSD/TEC used in its assessment of YT provider performance – namely, whether the person was in work on the day two months after finishing the programme. That test did not ask whether the work was full-time or part-time. Nor did it examine the “quality” of the job.
[8] The analysis in this paragraph gives the conclusion of the most recent report. Refer to Earle D (2018) Monitoring Youth Guarantee: Youth Guarantee Fees Free places Ministry of Education. Note that it’s possible, if unlikely, that the dial has shifted in the five years since the monitoring petered out.
[9] This monitoring was rigorous and of high integrity. But the comparison group isn’t a pure “control group”, as is used in clinical medical trials. It was selected using observable factors – age, ethnicity, gender, educational achievement and social and parental background. There may be differences between the two groups that can’t be measured – like personality traits, family influence, career and study preferences. So, the fact that the YG group had worse employment outcomes may not be entirely because of their participation in the programme.
[10] The three strands are YS: NEET programme, YS Youth Payment and YS Young Parent Payment.
[11] McLeod K, Dixon S and Crichton S (2016) Evaluation of the impact of Youth Service: Youth Payment and Young Parent Payment, Treasury Working Paper 16/07, The Treasury and Dixon S and Crichton S (2016) Evaluation of the impact of Youth Service: NEET programme, Treasury Working Paper 16/08, The Treasury
[12] See, for instance: Cunha and Heckman (2007); Carniero, Cunha and Heckman (2003), Kautz, Heckman et al (2015). For an overview and summary of the main findings of Heckman and his group, see Heckman (2011).
[13] Cunha and Heckman (2007)
[14] Heckman (2011)
[15] Cunha and Heckman (2007); Carniero, Cunha and Heckman (2003). See also Thomas et al (2019)
[16] Cunha and Heckman (2007)
[17] McGirr M and Earle D (2019) Not just about NEETs: A rapid review of evidence on what works for youth at risk of limited employment Ministry of Education
[18] MBIE et al (2023) Preparing all young people for satisfying and rewarding working lives MBIE, MSD, Ministry of Education, Ministry for Women
Strategy, Policy, Analysis | Tertiary Education 