PRODUCTIVITY, FIRM-LEVEL INTERVENTION & HIGH-QUALITY EVALUATION

Dr Tony Moody | Business & Science Group | BEIS

October 2016

Tony.Moody@beis.gov.uk

1

Belief: UK Productivity is being held back by deficiencies in management and leadership capabilities.

• Productivity decompositions attribute much of the productivity gap against G7 competitors to TFP, rather than labour quality or capital investment. Management & leadership capability would appear in TFP.

• International comparisons place the UK in mid-table for quality of its management and leadership.

• Deficiencies in UK management skills are a key driver of the productivity gap

• Bloom et al (2014) estimate that about a quarter of the UK’s productivity gap with the United States could be down to poor management

• Recent research with the ERC has expanded analysis to SMEs in the UK and finds shows similar links between skills, practices and performance (lower diagram), and a ‘long tail’ of firms with room for improvement

• UK Government invests extensively in improving workforce skills (schools, University, vocational quals). Likewise in improving capital markets, R&D, innovation. These are the ingredients of productivity.

• But investment in how the ingredients are combined – the recipe and the culinary skills – in UK is low, suggesting potential for intervention to valuably improve how people and resources are used more effectively.

How management skills and practices lead to improved outcomes

Source: Constraints on Developing UK Management Practices, BIS 2011

Source: BIS (2015) Leadership and management skills in small and medium-sized businesses.

Average management and leadership scores by country,

ranked from low to high

2

Hypothesis: Business-Level intervention can Improve Whole Economy Productivity

• Adding fertiliser to a plant to improve the ecology of a habitat.

• …Yet, so many processes between intervention and impact, and spanning big differences in scale

?

3

Where to start?! Remains extremely difficult to pick future HGFs

• Predicting future high-growth prone to Type I & Type II errors

• Only know the characteristics that improve the chances of future high growth:

• E.g. capabilities of owner/managers, level of ambition, market opportunities.

• …but often unobservable and even then hardly high precision.

• If hard to pick winning businesses, even harder to pick proven winning policy interventions!

4

Logic Model with Huge Black Box

Economy-level impacts

Displacement (factor market and/or product

market)?

Firm-level impacts

Perceptions or business metrics

Proxies for productivity.

Intervention

Selection effects? Purity of treatment

Firm to

receive

intervention

Productivity

Impact?

5

There is scarce robust evidence to measure impacts or improve design

• Evaluations typically focus on the beneficiaries’ assessment of benefits and a single policy design

• This approach does not provide a robust assessment

• The UK What Works Centre conducted a review of impact evaluations from across the OECD

• Just 1% provided credible evidence of growth impacts

Area Evaluations

Reviewed Robust approach

Growth impacts

identified

Business Advice 730 23 14

Access to Finance 1450 27 17

Innovation 1700 63 9

6

Randomised Control Trials (RCTs) of business advice in BEIS

• Using RCTs to address specific

questions in our hypothetical

causal chain.

• If one link in the circuit doesn’t

work, then whole black box

won’t work.

• Four RCTs on Business Advice:

- Main Growth Vouchers Programme (GVP)

- Growth Vouchers Additional Target group

- Business Schools RCT

- Growth Impact Pilot RCT

7

Growth Vouchers Programme - An example of a large scale trial

• UK launched the Growth

Vouchers Programme as a

research trial testing the

benefits of supporting

businesses to use external

strategic advice

• Between Jan 2014 and

March 2015:

• 38,000 applications

• 28,000 diagnostics

• 20,000 vouchers issued,

worth up to £39.4 million

• 6,400 vouchers used,

£11m claimed

• RCT implemented

successfully

8

Growth Vouchers Programme – Early Evidence

• Subsidising cost of Business Advice led to increased use of advice in treatment group (no evidence of substitution).

• Positive initial perception evidence from treatment group in terms of future business actions needed for growth.

• But too early for business performance metrics (and take a long time to measure in admin data because of reporting lags).

• There might be an adverse impact on self-assessment of business skills within treatment as they are exposed to what they don’t know, and consider focus (tackling ‘how’, not ‘what’).

9

Emerging lessons from wider BEIS evaluation

• Can’t pick likely future winners from data. We cannot identify future HGFs from standard business data on current and past characteristics (e.g. growth trajectories, size, age, sector, geography).

• …yet businesses receiving intervention grow quicker. Businesses participating on GrowthAccelerator (part of Business Growth Service) grew 3x faster than peers – so something going on – either successful selection based on richer data (inc. soft data), or an indicator of intervention impacts.

• Perception data from managers may not align to business performance.

• How businesses act on advice varies. We don’t know what changed behaviour will produce the most improvement.

• Time lags. Our wider evaluation evidence suggests that impacts on business metrics might take 3-5 years to reach full effect – just as many evaluations stop listening.

10

Many Questions remain unanswered

We don’t know:

• Improvement in SME-level productivity. [We measure turnover/jobs, not GVA]

• Improvement in wider economy productivity.

But suspect:

• Rather than measure productivity directly, past interventions seek to minimise substitution or displacement through eligibility and targeting.

• Return of investment unlikely to be scalable to offer impacts measureable at economy-wide level.

• …but they are likely to be a helpful investment with a speedy return because they engage currently economically active and ambitious businesses.

11

PRODUCTIVITY, FIRM-LEVEL INTERVENTION & HIGH-QUALITY EVALUATION

Dr Tony Moody | Business & Science Group | BEIS

October 2016

Tony.Moody@beis.gov.uk

Our research, evaluations and surveys are published here:

https://www.gov.uk/government/collections/enterprise-analysis-research

12