www.TheCIE.com.au
E X E C U T I V E S U M M A R Y
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
Delivering better health, research and jobs for Victorians
Prepared for
Department of Business and Innovation
14 February 2013
THE CENTRE FOR INTERNATIONAL ECONOMICS
www.TheCIE.com.au
COMMERCIAL IN CONFIDENCE
The Centre for International Economics is a private economic research agency that
provides professional, independent and timely analysis of international and domestic
events and policies.
TheCIE’s professional staff arrange, undertake and publish commissioned economic
research and analysis for industry, corporations, governments, international agencies
and individuals.
C A N B E R R A
Centre for International Economics
Ground Floor, 11 Lancaster Place
Majura Park
Canberra ACT 2609
GPO Box 2203
Canberra ACT Australia 2601
Telephone +61 2 6245 7800
Facsimile +61 2 6245 7888
Email [email protected]
Website www.TheCIE.com.au
S Y D N E Y
Centre for International Economics
Suite 1, Level 16, 1 York Street
Sydney NSW 2000
GPO Box 397
Sydney NSW Australia 2001
Telephone +61 2 9250 0800
Facsimile +61 2 9250 0888
Email [email protected]
Website www.TheCIE.com.au
DISCLAIMER
While TheCIE endeavours to provide reliable analysis and believes the material
it presents is accurate, it will not be liable for any party acting on such information.
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 3
www.TheCIE.com.au
Contents
Fkey findingFkey findingFkey findingFkey finding 1
Summary 4
Overview of Healthy Futures 4
Evaluation approach 6
Assessment of performance 7
Interim impacts for Victoria 12
Key evaluation messages from this review 12
Summary of key findings 17
BOXES, CHARTS AND TABLES
1 Stated objectives of the Healthy Futures program 5
2 Interim assessment of Healthy Futures 2006–2012 (Horizon 1) 8
3 Enhanced Performance Monitoring Framework tool 9
4 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheCIE.com.au
Summary
Healthy Futures is a $230.45 million Victorian Government capital investment to
enhance the health and economic wellbeing of Victorians and maintain the state’s
competitive strength in medical research.
Healthy Futures has enabled the building of a wide range of strategic capital works
that has provided physical space, workforce development and enabling technologies
for enhanced innovation. It has created opportunities to improve the quality of
medical research and contribute to the future health of Victorians.
As an interim evaluation, this assessment focuses on evaluating the outcomes and
impacts of Healthy Futures to date — or Horizon 1 (2006 – 2012). The interim
evaluation also attempts to identify early signs and indicators of impacts as well as
performance measures over the medium-term (Horizon 2) and thereafter (Horizon 3).
There is early evidence that the scale and quality of medical research has increased
as a result of Healthy Futures, and that more collaborative and translational research
behaviours are being embedded among researchers.
Overall, the projects have delivered on the outcomes anticipated to date, including
attraction of world- class researchers, expansion of training opportunities and access
to capital infrastructure and equipment, improved access to competitive funding,
increased use of platform technologies and collaboration through colocation. Tangible
signs of new outputs of research (health interventions, treatments and medicines and
associated downstream economic impacts) are expected in future Horizons.
Each of the Healthy Futures initiatives is well placed to deliver on their objectives,
understanding their contributory and facilitative role in achieving improved health
outcomes.
This interim evaluation draws together several findings in terms of what is already
working well and highlights important considerations for future assessment of the
suite of Healthy Futures initiatives.
Overview of Healthy Futures
The Victorian Life Sciences Statement: Healthy Futures (referred to hereon as Healthy Futures)
is a $230.45 million capital investment to enhance the health and economic wellbeing of
Victorians and maintain the state’s competitive strength in medical research. In essence,
Healthy Futures:
■ created new or expanded world class medical research infrastructure to expand
capacity and help attract the best and brightest people to generate commercial and
clinical opportunities from a world-class research base; and
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 5
www.TheCIE.com.au
■ enhanced access to health services and workforce, health information, and health
priorities. This access will ultimately contribute to access to new treatments for
Victorians — the best medicines, treatments and healthcare and hence to ensure they
continue to enjoy high quality health services.
The stated objectives of the Healthy Futures initiative at its time of development are
outlined in box 1.
1 Stated objectives of the Healthy Futures program
The Victorian Life Sciences Statement emphasises the ‘unique window of opportunity’ to
build upon existing strengths and take the next steps in medical research to solve
critical health problems. The following objectives of Healthy Futures are specified.
■ Capture new opportunities in areas of research where Victoria has critical mass
and a competitive advantage.
■ Attract investment and generate high quality jobs by maintaining and growing the
international reputation of Victoria’s research institutes.
■ Maximise opportunities for continued growth in national and international
collaborations and partnerships.
■ Deliver major benefits to business and industry by encouraging the
commercialisation of medical research.
■ Create a healthier future by enabling research to translate speedily into practical
health benefits for the entire community.
Healthy Futures captured an important moment in time, successfully capitalising on a
limited ‘window of opportunity’ in terms of the availability of financial capital and the
congruence of political will at the State and Commonwealth level. The Healthy Futures
investment was motivated by the need to address space constraints and overcrowding of
medical research infrastructure and to realise potential scale economies and increased
productivity of labour from increased collaborative research efforts.
It was also initiated during a period of substantial innovation investment by the Victorian
Government, reflecting a commitment to stimulate infrastructure and capability for
innovation to underpin long-term economic and social outcomes for the state. Healthy
Futures also sought to leverage additional philanthropic and Commonwealth funding
available given the favourable economic and financial landscape at the time.
While this creates challenges in attributing changes to Healthy Futures, it does mean that
the ultimate impacts associated with Healthy Futures are much larger than those that
would have been achieved in isolation.
In summary, Healthy Futures made several important contributions to the value
proposition for Victoria as being the place to do medical research in Australia, and
indeed the world. It also contributes to the broader long-term objective of boosting the
level of high quality medical research and associated clinical translations.
6 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheCIE.com.au
Evaluation approach
The evaluation methodology for Healthy Futures addresses the appropriateness,
effectiveness, and efficiency of the suite of projects within a program logic framework.
It spreads the program logic across three distinct time horizons given the long time lag
between research and workforce development, discovery, development, and translation
into clinical applications. In the case of Healthy Futures, the time lag is extended further
due to additional time required to secure the necessary resources associated with
capitalising effectively on infrastructure spending — which required additional leveraged
capital funding, leveraged operating funding, recruitment and purchasing before
additional research activities could get underway.
An innovative feature of this interim evaluation is that it tackles key challenges in
attribution for the Healthy Futures set of projects, particularly by incorporating qualitative
approaches to develop a ‘systems’ view of changes associated with the program. For
outcomes to date, inevitably many are associated and/or in some way linked to earlier or
subsequent measures. In addition, there is an extended payback period for these types of
investments and early signs of potential future impacts need to be assessed, despite
intensified challenges of attribution over the longer term.
The key methodological steps involved in this interim evaluation included the following.
■ Literature review — the information gathering phase reviewed relevant available
information regarding the ‘what’, ‘why’, and ‘how’ of the Healthy Futures initiative.
This included a review of Healthy Futures program documents and status updates,
Commonwealth and State Government policy statements, institutional annual
reports, independent evaluation reports and wider literature.
■ Stakeholder consultation — face–to–face meetings were held with each Healthy
Futures program beneficiary (for instance, various medical research institutes (MRIs),
providers of tertiary health services, CSIRO and other enabling medical research
infrastructure providers) and relevant Victorian government agencies.
■ Data collection — existing Healthy Futures evaluation surveys from 2008 to 2012 were
supplemented by open-ended stakeholder questioning. Bibliometric publications and
citations data, National Health and Medical Research Council (NHMRC) funding
data, Cooperative Research Centres (CRC) funding data, National Survey of
Research Commercialisation data (including regarding the scale of MRI patents,
licences and invention disclosures and the number of spin-off companies) and data
from the Victorian Government’s program of support for independent Medical
Research Institutes were also collected and analysed.
■ Quantitative analysis — Computable General Equilibrium modelling was undertaken
to estimate the indirect impacts of increased capital expenditure associated with
Healthy Futures to date.
■ Qualitative analysis — focus was placed on case studies of behavioural changes and
impacts, attracted expertise and associated new research outcomes, an assessment of
governance arrangements and horizons assessment of future impacts.
The methodological steps discussed above were used to undertake a horizons assessment
of Healthy Futures — meaning that each component of the program logic framework was
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 7
www.TheCIE.com.au
considered in the context of timing of incidence. As an interim evaluation, this
assessment focuses on evaluating Healthy Futures in the program logic framework over
Horizon 1 (that is, from 2006 to 2012 only).
The interim evaluation also attempts to identify early signs and indicators of impacts as
well as performance measures over the medium-term (Horizon 2) and thereafter
(Horizon 3). However, later evaluation of Healthy Futures will determine the full scope of
impacts and benefits over these future periods.
A summary of the appropriateness, efficiency and effectiveness of Healthy Futures as at
2012 is set out in chart 2.
Assessment of performance
The assessment above builds upon DBI’s existing Performance Monitoring Framework
to identify key indicators of program outcomes/impacts over various periods in order to
assess the appropriateness, effectiveness, and efficiency of the program elements. Chart 3
below summarizes the Performance Monitoring Framework developed by DBI for
Healthy Futures, and identifies additional assessment measures that were developed as
part of this interim evaluation.
The chart also shows which indicators are believed to be directly attributable to Healthy
Futures (pink) or partly attributable to Healthy Futures (black).
Assessment of Horizon 1 impacts demonstrates that Healthy Futures has made a genuine
difference to the scale and scope of medical research, workforce development and
enabling technologies in Victoria. All elements of the program are assessed to be largely
appropriate as a single event package of measures,1 although the extent to which they fill
gaps in existing capacity and enhance the productivity of research inevitably varies.
Healthy Futures has already delivered a range of economic and social returns to Victoria.
In addition, returns to date are expected to grow in future as the induced scale of
research, and changes in collaborative behaviours lead to increased research output,
clinical translations, and improved health outcomes. Indeed the outputs of Healthy
Futures are found to be critical to the probability of future success. Its value also lies in
improving the efficiency of other research and innovation spending in the State.
Healthy Futures projects have been positive enablers to achieving the overarching
objectives of the program, particularly in terms of positive workforce impacts, research
outputs and the branding value for Victoria as a destination for medical research.
1 ‘Single event’ refers to Healthy Futures as being a one-off investment in medical research
infrastructure. Hence, the positive assessment of Healthy Futures does not necessarily infer that
additional medical infrastructure investment would be warranted and/or would generate the
same benefit profile.
8
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheC
IE.com
.au
2 Interim assessment of Healthy Futures 2006–2012 (Horizon 1)
Ob
jecti
ves
Ob
jecti
ves
Ob
jecti
ves
Ob
jecti
ves
Horizon 1Horizon 1Horizon 1Horizon 1 2006 2006 2006 2006 ---- 2012201220122012 Financial Impact to DateFinancial Impact to DateFinancial Impact to DateFinancial Impact to Date CommentsCommentsCommentsComments
Inp
uts
Inp
uts
Inp
uts
Inp
uts
■ Total leveraged funding of
$508.1 million ($2.20 per public
$ invested)
■ Access to new equipment (e.g. bioprocessing facility) and enabling
technologies (e.g. VeRSI)
■ Access to new sources of operating expenditure from competitive funding
and philanthropic/other sources (HF seeded new funding sources). For
instance, the Australian Cancer Data Grid has been awarded independent
grants for ongoing operation (e.g. Roche, Sanofi Aventis, Novartis grants)
Assessment of appropriateness: Achieved
Ou
tpu
tsO
utp
uts
Ou
tpu
tsO
utp
uts
■ Increase Victoria GSP by about
$170 million per annum
■ Increase household
consumption by around $77
million per annum
■ 1480 employment years created
since 2006
■ Career paths for next generation researchers
■ Attraction of world-class international researchers . For instance, eight of
the twelve group leaders working at the ARMI were high profile
international recruits
■ Boost to scale, quality and functionality of research facilities (with benefits
in terms of greater collaboration and multi-disciplinary teams). For
instance, space constraints for WEHI researchers were solved and teams
were bought closer together into a common facility
■ Workforce enhancement via new training opportunities. For instance, new
research positions available at Healthy Futures institutes and more
medical university places available in rural and regional Victoria
Assessment of efficiency: Achieved
Ou
tco
me
sO
utc
om
es
Ou
tco
me
sO
utc
om
es
■ Increased clinical trials
providing increased income for
hospitals
■ Annual fee for service income of
over $1 million from industry
accruing to the Bioprocessing
facility
■ Maintenance of access to competitive Commonwealth research grants
(e.g. NHMRC, CRC, Commonwealth Education Investment Fund, ARC,
CSIRO)
■ More strategic prioritisation of research. For instance, the Victorian
Cancer Agency focuses on priority areas for improving health outcomes.
■ Providing for a faster, more efficient process to conduct clinical trials at
multiple sites
■ Upskilling the medical workforce
Imp
acts
Imp
acts
Imp
acts
Imp
acts
���� GSP impacts associated with enhanced expenditure from attracted labour from interstate/ overseas and additional capital investment
���� Small but positive increase in productivity of labour employed in medical research in Victoria
Assessment of effectiveness: Achieved
Healthy
Futures capital
funding
Attracted funding to complete
capital builds
New
equipment
Leveraged funding for operating
costs
Increased output and employment in the building and
construction phase
New platform technology
resources developed
New purpose-built facilities to support biomedical research and
capital improvements in
medical/research training
People involved in training courses on the use of Healthy
Futures funded infrastructure
Enabling facilities underpinning multiple institutions and
�access to equipment
Increased number of new medical training places
Access to ‘attracted’ world
class scientists
� capacity and uptake of PhD/Masters candidates under supervision in HF
supported activity
Increased medical and clinician
researchers, engineers and
support staff
Expatriate researchers and
clinician researchers
attracted back to Victoria
Steady/� access
to peer reviewed
(NHMRC/ARC)
research funding
Use of new
platform
technologies by
research groups
� collaboration
through
colocation
Capture new opportunities in
areas of research where
Victoria has critical mass and
a competitive advantage
Attract investment and generate high
quality jobs by maintaining and
growing the international reputation
of Victoria’s research institutes
Maximise opportunities for
continued growth in national and
international collaborations and
partnerships
Deliver major benefits to business
and industry by encouraging
the commercialisation of
medical research
Create a healthier future by
enabling research to translate
speedily into practical health
benefits for the entire community
Key
Achieved in full
Achieved as expected to date
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
9
www.TheC
IE.com
.au
3 Enhanced Performance Monitoring Framework tool
Indicators from DBIs Performance Monitoring Framework Additional indicators developed as part of this interim review
AP
PR
OP
RIA
TE
NE
SS
EF
FE
CTIV
EN
ES
S
EF
FIC
IEN
CY
LE
SS
ON
S
LE
AR
NE
D/
FU
TU
RE
DIR
CE
RTIO
N
Data source: The CIE
Evidence that medical research and training
capabilities were
inadequate
Evidence of potential loss of competitive
advantage
Consistency with DBI and broader Government
objectives
Impact on attraction and retention of
researchers, clinicians
and future leaders
Capacity to rapidly translate medical
research into improved
health outcomes
Maintaining/growing Victoria’s biomedical
research capabilities
Increase in quality jobs in innovative
industries
Increase in high quality
education and training
Recognition of Victoria as a world leader in biomedical research
and technology
Leveraging non-state Government funding for
facilities and
infrastructure
Attracting investment in and maximising
new research
opportunities
Enhancing the Victorian medical and research
training environment
Encouraging national/international linkages, partnerships
and collaboration
Increasing research focus on clinical / commercial
outcomes
Enhancing capacity, efficiency and
integration of existing facilities
Lessons from delivery of Healthy Futures
to date
Opportunities for future improvement in design
and delivery of
Healthy Futures
Adequacy of resources to enable Healthy Futures to achieve its
objectives
Efficiency of infrastructure
delivery
Efficiency in the use of
DBI resources
Efficiency of administrative
costs
Alignment with Commonwealth
Government objectives, policy and
investments
Alignment with the goals of
non–Government
stakeholders
Focus on additionality
(non-duplicative and
game changing)
Focus on system-wide
performance
Alignment with best practice research
trends
Primarily support for basic, pre-commercialisation
and publicly funded
research
Investment unlikely to crowd out commercial activity given market
failures
Creation of career paths for the next generation
of researchers in
Victoria
Impact on access to
new equipment
Direct economic benefits — Victorian GSP, employment, household
consumption
Strengthen Victoria’s reputation and international
presence
Attract competitive funding, industry/philanthropic support for operating
expenditure
Achieving scale and
integration
Opportunity for prioritisation of
research activity
Increase industry partnerships, spin-off
companies and
commercialisation activity
Increase clinical trial
activity in Victoria
Key
Evidence of the impact directly attributable to Healthy Futures
Evidence of the impact partly attributable to Healthy Futures
10 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheCIE.com.au
In terms of outputs:
■ Healthy Futures has already realised positive impacts on access to new equipment, both
directly and indirectly. For instance, Healthy Futures has directly provided Victorian
researchers access to a bioprocessing facility (creating a substantial private revenue
stream of $1.2 million annually by making access open to industry); equipment
required for research using bioresources (at a bioresources facility established for the
Austin Biomedical Alliance); and New Grid infrastructure. Indirectly, the impending
availability of new infrastructure and larger research capacity provided the impetus
and business case for advanced biomedical imaging equipment (Victorian Biomedical
Imaging Capability) in facilities across Melbourne, including the new Neuroscience
facilities;
■ in the majority of cases, Healthy Futures has been an effective recruitment tool and has
assisted to meet workforce challenges across the spectrum of need. It has already been
successful in assisting Victoria to attract world-class researchers. For instance, eight of
the twelve group leaders now working at the Australian Regenerative Medicine
Institute (ARMI) were high profile international recruits. In addition, the Walter and
Eliza Hall Institute (WEHI) has attracted back key research staff after various stints in
overseas research institutes following Healthy Futures investments. Healthy Futures has
also better enabled the creation of career paths for the next generation of researchers
in Victoria by enhancing the quality and quantity of research opportunities;
■ Healthy Futures has improved workforce development by facilitating improvements in
the quality of training and development, including enhancements to facilities in rural
and regional areas. For instance, the Medical University Places Capital Infrastructure
program built teaching, training and research facilities, tutorial rooms, libraries,
student accommodation and other education amenities to a) improve opportunities
for medical training in regional Victoria, and b) encourage medical graduates to work
in rural areas. In addition, increased physical capacity at Healthy Futures medical
research institutes has increased the number of research training positions available.
For instance, the WEHI currently has 80 PhD students, of which some are directly
attributable to the additional physical space that has resulted from Healthy Futures;
■ as well as bringing direct economic benefits through net capital inflow, Healthy Futures
also strengthens Victoria’s international presence and reputation as a world-class
destination for medical research. Increased reputation and leadership of Victorian
research is evidenced by, for instance, by the ARMI being selected to host the
European Molecular Biology Laboratory (EMBL) Australia. Associate Membership
with this global centre of excellence2 provides Australian researchers with access to
state-of-the-art infrastructure and technologies, expertise and networks in Europe; new
opportunities to forge collaborations with European researchers and institutions; and
access to European funding opportunities. ARMI has also recently attracted a
partnership with Japan’s Systems Biology Institute highlighting the ARMI’s success in
forging strong links with internationally recognised research organisations.
■ Healthy Futures has improved opportunities for clinical translation. Clinical practice
aligned and integrated with discovery research is a strong feature of research culture
2 EMBL is the most cited scientific institution outside of the USA in molecular biology and
genetics, with an extremely high impact of an average of 51.9 citations per paper.
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 11
www.TheCIE.com.au
and practice, which underpins Victoria’s reputation and performance in health
sciences. This was given a significant boost through the Healthy Futures investment.
– At the WEHI, new facilities have enabled much closer interaction with clinicians
at the Royal Melbourne Hospital. In addition to direct benefits resulting from more
rapid and efficient translation of research findings to clinical practice, this new
association has encouraged a more active research interest amongst clinicians,
which in turn is likely to result in improved clinical care and outcomes for patients.
– Further, a new test developed by researchers at the Burnet Institute offers a simpler
and cheaper way of monitoring HIV in sufferers and a wristwatch that
continuously monitors the health of patients with Parkinson’s disease developed by
Australian Centre for Neuroscience and Mental Health Research (ACNMHR)
scientists was announced at the international BIO2009 Conference.
At this relatively early stage, Healthy Futures has achieved several successful outcomes:
■ it has achieved scale and integration, which has helped grant recipients continue to
attract major national and international competitive funding, as well as industry and
philanthropic support;
■ it has enhanced collaborations on new ‘platforms’ underpinning future medical
science outcomes;
■ it has provided the opportunity for prioritisation of research activity following
integration of research institutions and teams. For instance, investment by Healthy
Futures in the Victorian Cancer Agency (VCA) enabled a coordinated approach on
strategy for cancer research in Victoria. Its strategy is increasingly to focus on priority
areas for improving health outcomes. The strategy includes initiating new research
areas where there is clear evidence that current research effort is misaligned relative to
priority health issues;
■ it has led to some early signals of success in indicators of clinical translation of
research efforts of MRIs including new and pre-existing international patents and
income from clinical research projects; and
■ there are also positive developments in clinical trial activity that are partly attributable
to Healthy Futures. The streamlining of ethics approval for multi-site trials has directly
addressed time and cost factors that influence trial location decisions for multinational
pharmaceutical companies and multi-site clinical trial activity in Victoria over the last
few years. This will have been at least partly facilitated by Healthy Futures. Data from
the Australian and New Zealand Clinical Trials Registry shows an increasing trend in
the number of registered trials occurring at the broader state level.
To date there are few early examples of industry partnerships induced by Healthy Futures.
An exception is Victorian technology company Global Kinetics Corporation, which
worked closely with researchers from the Florey Neuroscience Institutes to develop a
prototype micro-electro-mechanical wristband device. It is expected that spin off impacts
for biotechnology start ups are expected to increase and evolve in a way that is at least
partly attributable to various Healthy Futures projects. This is because there is a strong
supply side element to attracting downstream activity in medical research in Victoria.
Biotechnology start–ups and existing biotech and pharmaceutical companies are attracted
by a strong supply base of people, skills, and intellectual property — all of which have
been enhanced by Healthy Futures.
12 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheCIE.com.au
It is also expected that the increased reputation brand and leadership of Victoria’s
medical research sector will be evidenced in the future by additional extramural staff
attraction and in rising research income sourced internationally. Further, the quality and
quantity of Victorian research outputs are expected to continue to rise evidenced by
increasing publication rates and citations received by Victorian researchers.
Interim impacts for Victoria
Healthy Futures seed funded a total program spend of $932 million in Victoria, leveraging
over $701 million from other Government and non-Government sources that is unlikely
to have been invested without Healthy Futures.
On average, Healthy Futures funding leveraged $4 for every dollar invested by the
Victorian Government, which varied across projects from $1.20 to $18.60 per dollar
invested. Including financing from the Victorian Government in addition to Healthy
Futures, each dollar of state government funding leveraged $2.20 from Commonwealth,
philanthropic, institutional and other sources.
Based on economy-wide modelling on Victoria, by 2011-12, Healthy Futures is estimated
to have increased Victorian GSP and household consumption by approximately
$170 million and $77 million respectively. Of this, $81 million represents the return to
labour, which based on average wage rates in Victoria, is equivalent to close to 1 480
employment years created. Large future economic impacts are expected over subsequent
Horizons, which are expected to show more direct benefits in terms of research
collaboration, industry partnerships, and ultimately improved clinical treatments and
therapies that will improve health outcomes.
■ Healthy Futures has made a strategic and lasting impact on the scale and quality of
research activity and behaviours to make it more likely that successful clinical
translations will result over time, producing improved health outcomes over the
medium to longer term.
■ Given the nature of drug development and the need for MRI initiated research to be
developed by start up companies and/or downstream users, it is highly likely that
Healthy Futures has and will generate positive impacts in terms of spin off companies
over time.
■ In terms of future research outputs, Horizon 2 is expected to witness an increase in the
number of international publications in peer reviewed journals, while in Horizon 3,
the number of citations attracted by Victorian-authored publications is expected
indicating the rising quality of research outputs.
In essence, Healthy Futures is expected to have made a lasting impact to the foundation
that matters in terms of probability of successful, outcomes based research.
Key evaluation messages from this review
The interim evaluation of Healthy Futures is a positive one and there are findings that
reinforce positive elements of the program and serve to highlight areas where future effort
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 13
www.TheCIE.com.au
and resources may need to be brought to bear in order to best ensure that the potential
benefits from the investment are realised.
Governance and administration
Projects of the size of Healthy Futures take considerable resources to govern and
administer well. The following levels of governance are recognised as being important in
assessing Healthy Futures:
■ project — implementation and subsequent management for each Healthy Futures
initiative, particularly the delivery of intended outputs — this is the responsibility of
the leaders of the institutions granted Healthy Futures funding (e.g. WEHI, VCA);
■ program — oversight and review of the performance, outcomes and impacts for the
overall Healthy Futures program — this is the responsibility of the DBI; and
■ policy — oversight and review of the appropriateness and effectiveness of Healthy
Futures program in relation to higher-level, longer-term, and wider-ranging
government goals for health and medical sciences in Victoria — this is the
responsibility of the Victorian Government.
Project
Healthy Futures has a ‘project’ governance component, given that various discrete projects
each have their own strategic intent and high level governance structure in place. There
are examples where this project level of strategic governance was very sound. The WEHI
was an exemplar in this regard, complementing overall governance through its
Board/Audit and Risk Committee with a dedicated project oversight committee. The
high-level New Building Sub-Committee met monthly over the course of the project. The
Sub-Committee received full executive support from the Institute and was attended by
key project consultants (project management, architects, and quantity surveyors) and
additional consultants as required. The Sub-Committee also included opportunities for
DBI and Department of Health participation from the outset. In additional to steering the
capital development, the Sub-Committee monitored the Institute's obligations to funders
of the project and compliance with all reporting requirements. This approach ensured
strong governance for the specific project and strengthened communication with DBI and
its higher-level (program) governance role.
Project governance was arguably less strong with respect to the Australian Cancer Data
Grid project, where there could have been stronger emphasis on business planning,
strategic intent in the activities undertaken and focus on bringing the new platform to
end-users. Greater emphasis on these elements should assist with building broader
stakeholder support and more widespread use of project outputs.
In terms of enhanced project level governance, in addition to regular reporting to DBI,
leaders of the respective projects within the Healthy Futures program could be involved in
more regular and collaborative self-review, with reflections shared among program
beneficiaries. This would encourage active learning across the Healthy Futures
‘community’, through which governance of individual projects should improve over
time.
14 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheCIE.com.au
Program
Healthy Futures was a grants-based program, with much of the project management the
responsibility of medical research institutes and agencies.
DBI’s program governance role (oversight and review of performance, outcomes and
impacts of the overall Healthy Futures program) is considered appropriate. DBI had
generally sound strategic governance structures in place for interacting with each Healthy
Futures project, with interdepartmental consultation established wherever relevant,
arranged as required on an as-needed basis. These arrangements generally enabled DBI
to be close enough to projects to genuinely ensure their most appropriate, effective and
efficient implementation, with some qualification relating specifically to the
administrative resources allocated to project oversight.
With no specific allowance made in the program budget for government administration,
in-kind government resources were likely to have been spread too thin. The total
administrative costs of the Healthy Futures program were estimated by DBI to be
$1.66 million since program inception to June 2012. This represents 0.7 per cent of the
Healthy Futures investment of $230.45 million and 0.4 per cent of total program costs
when subsequent Victorian Government funding is included. We note however, the
substantial in-kind support that was mobilised by the funding recipients to deliver Healthy
Futures projects.
Australian National Audit Office (ANAO) suggests that ‘there is no apparent benchmark
for the ratio of administrative costs to program costs. Costs observed in audits of grant
programs range from as low as 1 per cent to as high as 35 per cent’.3 The appropriate
share of administrative costs to program funds will vary according to:
■ the size of the program — smaller programs will have a higher proportion of fixed
costs, which will raise the proportion of administrative costs to program funds; and
■ the risk associated with the program — higher risk individual projects or programs
will require more costly risk treatments including closer monitoring.
Given these factors, it is likely that the administrative resources allocated to Healthy
Futures were too limited. The ANAO suggests that ‘insufficient administrative resources
to manage grant programs increase the risk that the program’s objectives may not be
achieved in an efficient, effective and timely manner’. That said, administrative resources
do appear to have been managed effectively for contract management and project
oversight activities.
DBI as lead agency also undertook extensive reporting to Cabinet, in partnership with
the Department of Health. This was particularly the case in the period from inception to
late 2010, which was a critical period in the development of each initiative. No criticisms
were raised by any stakeholders on the role of DBI in managing and reviewing any of the
Healthy Futures projects.
3 Australian National Audit Office 2002, Administration of Grants: Better Practice Guide, p.14.
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 15
www.TheCIE.com.au
Policy
Healthy Futures is a good example of governance working well at the ‘policy’ level.
Healthy Futures is a highly strategic set of initiatives that are well complemented by, and
consistent with, the broader innovation framework of the Victorian and Commonwealth
Governments. Consistent with the Commonwealth’s innovation strategy Powering Ideas
and the Victorian Government’s recent Technology Plans for the Future, Healthy Futures
focuses on enhancing Victoria’s competitive strength and reputation in science and
innovation and continued priority investment in medical research. Healthy Futures had
longer-term expectations in mind, which were captured in the specific objectives defined
at the time of program inception, and have continued to be referred to by program
beneficiaries when reflecting on Healthy Futures today.
The importance of enabling ‘system’ evaluation capability in future
Longer-term impacts of a program such as Healthy Futures will be dispersed through the
medical sciences ‘system’ in Victoria. Future evaluation of Healthy Futures should capture
if, and how, the ‘system’ performance targets are being met over time. This is particularly
important given that impacts genuinely attributable to Healthy Futures are likely to be
small and understate their true value.
Improved data collection could help to achieve this, possibly facilitated by more
administrative resources being dedicated to project oversight. For instance, not all
recipients provided evaluation surveys to DBI and in most cases, responses were
incomplete. While we note that the data collection tool was less relevant to some
projects, while others were subject to separate and specific program evaluations,
incomplete survey data has created challenges for this evaluation and will continue to do
so. It is therefore important that greater compliance with reporting responsibilities be
achieved.
Further, more detailed and timely information on medical science, public health and
business innovation metrics for Victoria, to help assess changes in priority policy areas
over time. It is also important to focus on ‘drivers’ of medical sciences performance and
outcomes.
For example, bibliometric assessments of research outputs and their impact — by subject
area and institution — can be valuable for understanding the international
competitiveness of research capability in Victoria and therefore the potential to compete
beyond Victoria for research funding. Similarly, business innovation metrics for firms
involved in developing and deploying medical technologies derived from research in
Victoria’s medical research institutes can help understand constraints and opportunities
affecting business growth.
The current Healthy Futures survey tool is well designed and appropriate. However,
incomplete responses to date limit their usefulness for program evaluation and data
analysis purposes. In order to decrease the compliance burden and increase response
rates, consideration may be given to sourcing data through existing structures. For
instance, MRIs could be asked to provide responses to the National Survey of Research
Commercialisation and elaborate where deemed appropriate. Further, bibliometric data
16 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheCIE.com.au
may be sourced from independent sources, which would reduce the compliance burden
on MRIs, and also allow for comparative data analysis.
Options for future investment should be explored
Collective effort to increase access to operational funding is likely to be needed to realise
greater value from the infrastructure investments made through Healthy Futures.4 This is
likely to depend on leadership to facilitate a collaborative effort for identifying and
capitalising on funding options (including Commonwealth and international funding
agencies, commercial and philanthropic investors), thereby strengthening Victoria’s
overall competitiveness in attracting such investment.
Securing ongoing operational funding is an important issue, although it is not a direct
issue for this evaluation. All stakeholders were clear during the development of the
initiatives that operating funding was not provided for under Healthy Futures and all
stakeholders agreed that operating funding would be obtained from existing funding
sources.
Operating funding is an issue being considered in the McKeon Strategic Review of Health
and Medical Research in Australia, which advocates that MRIs receive at least 60 per cent
indirect cost loading for national competitive grants. In its response to this consultation
paper, the Victorian Government emphasised the importance of predictability and
certainty of funding for health and medical researchers over the longer-term. Consistent
with this view, it is important to articulate what the investment options are for building
on Healthy Futures achievements to date.
Consideration should also be given to encouraging and rewarding successful
implementation. As most Healthy Futures projects are still at relatively early stages
(relative to their longer-term value), there is plenty of scope to influence the overall
performance of the investments made.
Further State Government investment in medical sciences should therefore encourage
and reward project recipients to maximise the impact of the (sunk) investment already
made. This could include on-going expectations that project recipients define what is
being done, and what more is needed to maximise long-term value, as part of future
funding requests related to the infrastructure established through Healthy Futures.
Tell the Healthy Futures Story
It is important to promote the success and value of the strategic investment in Healthy
Futures. As investment priorities and mechanisms vary over time, there is merit in
ensuring a record of how different approaches work, at different stages of sector
development and maturity.
4 Operational funding may refer to a) indirect cost funding, b) direct research funding from non-
Victorian Government sources and c) equipment and infrastructure funding. Future references
to operational funding include all of the above.
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 17
www.TheCIE.com.au
Healthy Futures was implemented during a period of substantial infrastructure investment
in medical sciences and other innovation areas in Victoria. The aggregate impact of
these investments is inevitably complex and hard to track over time.
This means it may be difficult to ensure future policy makers appreciate critical features
of this investment approach. The Healthy Futures story should therefore be recorded in a
format that is easily absorbed by future policy audiences. This interim evaluation of
Healthy Futures will be important in this regard.
Summary of key findings
■ Key finding #1: Healthy Futures seed funded a total program spend of
$932.3 million, leveraging over $508 million from non-Victorian Government
sources that is unlikely to have been invested without Healthy Futures.
■ Key finding #2: On average, Healthy Futures and subsequent Victorian
Government funding leveraged $2.20 from Commonwealth, philanthropic,
institutional and other sources for every dollar invested by the State Government.
This varied across projects from $1.00 to $4.40 per dollar invested.
■ Key finding #3: Healthy Futures has had a positive impact on access to new
equipment, both directly and indirectly. In the case of the bioprocessing facility, a
private revenue stream has already been created. While this is a positive outcome
and demonstrates private value, it remains the case that Healthy Futures
investments were not intended to establish profit-making centres, but rather to
support and advance Victoria’s medical research base.
■ Key finding #4: By 2011-12, Healthy Futures is estimated to have increased
Victorian GSP and household consumption by approximately $170 million and
$77 million respectively. Of this, $81 million represents the return to labour,
which based on average wage rates in Victoria, is equivalent to close to 1 480
employment years created.
■ Key finding #5: Healthy Futures has enhanced the quality and quantity of research
training opportunities and enabled career paths for early career researchers in
Victoria, and has significantly increased the capacity for regional medical training.
■ Key finding #6: Healthy Futures has already been successful in assisting Victoria
to attract world-class researchers. As well as bringing direct economic benefits
through net capital inflow, it also strengthens Victoria’s international presence
and reputation.
■ Key finding #7: Healthy Futures has enhanced collaborative approaches to
research, including through e-Research and ICT-enabled platforms and has better
enabled clinicians to be engaged with research.
■ Key finding #8: In the few years since Healthy Futures investments became
operational, Victoria has been able to maintain its ‘first-place’ status in terms of
access to competitive Commonwealth research grants.
18 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures
www.TheCIE.com.au
■ Key finding #9: Healthy Futures was an important first step in helping to improve
the viability of a clinical research environment in Victoria. The streamlining of
ethics approval for multi-site trials has directly addressed critical time and cost
factors that influence trial location decisions for multinational pharmaceutical
companies and multi-site clinical trial activity in Victoria over the last few years
will have been at least partly facilitated by Healthy Futures.
■ Key finding #10: Healthy Futures has had an important impact on supporting the
conditions for future positive translations from research to clinical practice and
treatments.
■ Key finding #11: Healthy Futures has made a strategic and lasting impact on the
scale and quality of research activity, and behaviours it has reinforced make it
more likely that successful clinical translations will result. It is the finding of this
evaluation that improved health outcomes will be a result of the program over the
medium to longer term (Horizons 2 and 3).
■ Key finding #12: Given the nature of drug development and the need for MRI
initiated research to be developed by start up companies and/or downstream
users, it is highly likely that Healthy Futures will generate positive impacts in
terms of spin off companies.
■ Key finding #13: The quantity and quality of Victorian research outputs have been
rising steadily since 2002 and indicate that, in terms of research output, Victoria
is currently ‘punching above its weight’. Healthy Futures is expected to reinforce
this trend. Horizon 2 is expected to witness an increase in the number of
international publications in peer reviewed journals, while in Horizon 3, the
number of citations attracted by Victorian-authored publications is expected,
which would indicate the rising quality of research outputs.
■ Key finding #14: Healthy Futures is expected to have an important impact on the
international profile, reputation and leadership of Victorian research. Healthy
Futures initiatives have already attracted world class researchers and peaked the
interest of global centres of excellence such as European Molecular Biology
Laboratory and Japan’s Systems Biology Institute.
■ Key finding #15: The Department of Business and Innovation should continue to
promote and facilitate the linkages between medical research institutes and
private industry in order to help institutes diversify their funding base.
Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 19
www.TheCIE.com.au
THE CENTRE FOR INTERNATIONAL ECONOMICS
www.TheCIE.com.au