Date post: | 06-Feb-2018 |
Category: |
Documents |
Upload: | truongmien |
View: | 215 times |
Download: | 1 times |
Evaluation of the Living Libraries Infrastructure Program Local Government VictoriaDepartment of Environment, Land, Water and Planning
3 February 2015
Contents
Contents........................................................2Executive summary........................................3
Background............................................................................................3Findings and recommendations.............................................................4
Introduction...................................................6Overview................................................................................................6Evaluation scope....................................................................................6Outcomes logic model.......................................................................8Evaluation methodology and data sources............................................8
Interviews...................................................................................................9Survey........................................................................................................9
Evaluation limitations.............................................................................9Findings........................................................10
Program delivery – summary data.......................................................10Appropriateness and relevance............................................................12
Key evaluation questions..........................................................................12Findings....................................................................................................12
Effectiveness........................................................................................15Key evaluation questions..........................................................................15Findings..................................................................................................15
Initiative delivery and efficiency..........................................................18Key evaluation questions....................................................................18Findings....................................................................................................18
Lessons and future directions..............................................................20Lessons on initiative design and delivery.........................................20Impact of stopping the initiative...............................................................21Conclusions and recommendations.................22
Conclusions..........................................................................................22Recommendations...............................................................................22
Attachments..................................................23Attachment 1: Evaluation design.........................................................23Attachment 2: Outcomes logic model..................................................30Attachment 3: Interview guide.............................................................31Attachment 4: Survey instrument........................................................33Attachment 5: Survey data..................................................................38Attachment 6: References....................................................................62Attachment 7: Australian Library and Information Association Statement on Public Library Services.........................................................................63Attachment 8: Australian National Audit Office Principles for Better Practice Grant Processes...................................................................................64
Living Libraries Infrastructure Program Evaluation Page 2 of 65
Executive summary
Background
The Living Libraries Infrastructure Program is a four-year grants program designed to assist regional libraries and local governments to provide high-quality and accessible public library infrastructure that meets the changing needs of communities. These grants are additional to and separate from core service funding to the library services by the state government.
The Living Libraries Infrastructure Program’s objectives are to:
• provide new or improved public library infrastructure;
• support the role of public libraries in strengthening communities and encouraging opportunities for community participation;
• encourage and create lifelong learning opportunities for Victorian communities; and
• facilitate free access to information and reading resources.
Grants range up to a maximum of $750,000. Around fifteen projects are approved each year. Sixty-three projects have been approved over the last four years, with $17.2m of LLIP grants contributing to projects of $180m total worth.
Local Government Victoria has commissioned an evaluation to provide information on the effectiveness of the Living Libraries Infrastructure Program and determine whether the program has met the appropriate policy objectives, planned outputs and desired outcomes. The evaluation is also to inform the government's current and future policy and decision-making around the program; a new business case will be developed early in 2015 for a proposed new round of the program.
The evaluation has assessed the program against the requirements of the Department of Treasury and Finance’s Evaluation Policy and Standards for Lapsing Programs:
• justification of the program and links to policy;• effectiveness;• funding/delivery;• risk; and• efficiency.
Evidence for the evaluation was gathered in December 2014 and January 2015. Evidence sources included:
• document and data reviews;• key informant interviews; and • an online survey of library service managers.
Findings and recommendations
Findings
Living Libraries Infrastructure Program Evaluation Page 3 of 65
The key findings included the following:
• The 17.2m of LLIP grants was evenly divided between metro projects ($8.1m, 47% of LLIP grants) and regional projects ($9.1m, 53%).
• The LLIP proportion of total projects’ expenditure was higher for regional projects (22% of total projects’ expenditure) than metro projects (6%).
• Stakeholders interviewed and surveyed for the evaluation were overwhelmingly in favour of the program and were unanimous in saying that much public library infrastructure would remain unbuilt or left in a dilapidated state in its absence.
• Grant processes were said to be open, relatively transparent, and easy to comply with. Local Government Victoria was praised for running a collaborative and simple grants system.
• Libraries that have implemented infrastructure projects with LLIP grants assistance report greater accessibility of services, increased usage rates and higher user satisfaction ratings as a result.
• The increasing cost of buildings means that the maximum grant amount of $750,000 is diminishing in significance compared to the total cost of a major new library building.
• Stakeholders interviewed and surveyed for the evaluation described the range of project sizes and scope funded by LLIP as being highly appropriate, as it responds to the range of needs across the sector.
Conclusions
The Living Libraries Infrastructure Program (LLIP) is a grant funding initiative that is seen by Victorian public library services as a vital and integral part of the sector’s annual infrastructure funding. It contributes around 10% on average of public libraries’ infrastructure spending and is considered to play a vital role in locking in counterpart funding from local Councils.
Grant processes are viewed by applicants as efficient, open and transparent, simple to comply with, and delivered in a collaborative manner between the Department and public library services.
Recommendations
It is recommended that:
1. The Department review the maximum grant size.
2. The Department review the rationale for deciding the limitations to non-fixed infrastructure for grant funding.
3. The Department review whether there is a case for articulating decision making guidelines on a policy basis for annual division of the grant funds pool between large, medium and small projects.
Living Libraries Infrastructure Program Evaluation Page 4 of 65
4. The Department articulate the grant decision-making principles in the guidelines made available to applicants.
5. The Department consider changing the timing of the grant process to better fit library services’ capital budget planning processes, for example by running multiple grant rounds each year or a continuously open round. There may be merit in running separate processes for large and small projects.
Living Libraries Infrastructure Program Evaluation Page 5 of 65
Introduction
Overview
Local Government Victoria supports public libraries through a range of grants and other funding and is also responsible for governance issues relating to regional library corporations and Mechanics' Institutes.
Local Government Victoria administers the Government's grants to libraries, including managing three year funding and service agreements with:
thirteen regional library services (each serving two or more councils);
thirty-three single-council library services; and
a state-wide library services for people with print disabilities.
These funding and service agreements cover two program areas:
core funding, which helps councils provide public library services; and
local priorities funding, which supports specific local services identified by the library as a priority.
The Living Libraries Infrastructure Program is a four-year grants program designed to assist regional libraries and local governments to provide high-quality and accessible public library infrastructure that meets the changing needs of communities. These grants are additional to and separate from core service funding to the library services by the state government.
The Living Libraries Infrastructure Program’s objectives are to:
provide new or improved public library infrastructure; support the role of public libraries in strengthening communities and encouraging
opportunities for community participation; encourage and create lifelong learning opportunities for Victorian communities;
and facilitate free access to information and reading resources.
Grants range up to a maximum of $750,000. Around fifteen projects are approved each year. The sixty-three projects approved over the last four years are at varying stages of completion.
Evaluation scope
Local Government Victoria has commissioned an evaluation to provide information on the effectiveness of the Living Libraries Infrastructure Program and determine whether the program has met the appropriate policy objectives, planned outputs and desired outcomes. The evaluation is also to inform the government's current and future policy and decision-making around the program; a new business case will be developed early in 2015 for a proposed new round of the program.
The evaluation has assessed the program against the requirements of the Department of Treasury and Finance’s Evaluation Policy and Standards for Lapsing Programs:
Living Libraries Infrastructure Program Evaluation Page 6 of 65
justification of the program and links to policy; effectiveness; funding/delivery; risk; and efficiency.
The top level questions and sub-questions have been provided in the brief, based on DTF requirements:
1. Justification / problem: What is the evidence of a continued need for the program? What is the role for government in delivering this program?
To what extent does the Living Libraries Infrastructure Program continue to address a demonstrable need and is responsive to the needs of Victorians?
To what extent have options been investigated to address the identified need or problem?
Does the Living Libraries Infrastructure Program continue to be the best way to respond to the problem and deliver the intended outcomes?
How have the economic, environmental and social conditions changed since the program was funded and how will continuation of the program meet these conditions?
2. Effectiveness: What is the evidence of the program’s progress toward its stated objectives and expected outcomes, including the alignment between program, its outputs (as outlined in Budget Paper 3), departmental objectives and any stated government priorities?
To what extent has Living Libraries Infrastructure Program progressed towards its stated objectives and outcomes it was seeking to achieve (at start-up and any revisions)?
Why was this program approach considered the best way to achieve the outcomes?
3. Funding / delivery: (a) Has the program been delivered within its scope, budget, within expected timeframes, and in line with appropriate governance and risk management practices; and b) Has the department demonstrated efficiency and economy in relation to the delivery of the program?
To what extent has Living Libraries Infrastructure Program been delivered within its scope?
To what extent has Living Libraries Infrastructure Program being delivered within its budget?
To what extent has Living Libraries Infrastructure Program been delivered within the expected timeframe?
To what extent has Living Libraries Infrastructure Program been delivered in line with appropriate governance and risk management practices?
Living Libraries Infrastructure Program Evaluation Page 7 of 65
To what extent has Living Libraries Infrastructure Program been delivered at lowest possible cost without compromising quality?
4. Risk: What would be the impact of ceasing the program?
What would the impact be if DTPLI was to successfully exit from delivering Living Libraries Infrastructure Program if the government so desired?
What strategies have been identified to minimise any negative impacts of this exit?
5. Efficiency: What efficiencies could be realised?
If ongoing funding was provided, what level of efficiencies could be realised?
Outcomes logic model
During the preparation of the evaluation plan the evaluator, in collaboration with LGV, developed an Outcomes Logic Model (OLM) for the LLIP. The OLM is attached at Attachment 2.
The OLM describes the program logic in a table to illustrate the links between policy, strategies, inputs, activities, outputs and outcomes.
The OLM provides a guide to the evaluation design and data collection by illustrating how the program is intended to work in producing outcomes for the beneficiaries and the community.
Evaluation methodology and data sources
The evaluation design is attached at Attachment 1. In brief the approach is as follows.
1. Develop Outcomes Logic Model2. Develop evaluation framework & key questions3. Develop interview questions and guide4. Develop survey questionnaire5. Collect data:
a. Performance data, including PLVN annual performance statistics.b. Key informant interviews of ten library managers.c. Online survey of library services.
6. Analyse7. Report
Living Libraries Infrastructure Program Evaluation Page 8 of 65
Interviews
Key informant interviews were conducted in December 2014 of ten library service managers selected at random1 from the list of grant recipients.
Survey
An online survey was run in January 2015. Forty-two library service managers responded, a 91% return rate.
Evaluation limitations
The evaluation has worked within several constraints:
1. The time for the evaluation research and analysis is limited (one working month).2. Evidence of the program’s long-term impact is necessarily indirect, as many
factors contribute to the creation of strong socially-inclusive, connected and active communities where people like to live, the LLIP being only one of them.
1 Grant recipients were assigned a sequence number and a random number generator app was used to select candidates for interview.
Living Libraries Infrastructure Program Evaluation Page 9 of 65
Findings
Program delivery – summary data
The LLIP has delivered $17.2 m of funding through sixty-three grants over the period 2011-14. Summary data is shown in below.
Table 1: Funding summary 2011-14
Year Number of applications
Number of grants made
Amount requested
Amount granted
2011 15 10 $5.2 m $3.4 m
2012 27 22 $10.8 m $5.5 m
2013 20 13 $8.6 m $3.5 m
2014 24 18 $8.5 m $4.8 m
TOTAL 86 63 $33.1 m $17.2 m
Significant competition for the grant funding is illustrated by the fact that applications were made for an amount of grant money almost double that available. At the same time, three-quarters of grant applications by number were successful.
The size of grants ranged from $6,000 to the maximum amount under the scheme, $750,000.
<$100,000
$100,000 to $199,999
$200,000 to $299,999
$300,000 to $399,999
$400,000 to $499,999
$500,000 to $599,999
$600,000 to $699,999
>$700,0000
5
10
15
20
25
Histogram of all LLIP grants funded 2011-2014
Size of grant
Num
ber o
f gra
nts
Figure 1: Distribution of grant size – LLIP grants 2011-2014
Table 2, over, shows projects funded by type.
Living Libraries Infrastructure Program Evaluation Page 10 of 65
Table 2: Projects funded, by type, 2011-2014
Project Type Number
New Library/Relocation 23
Redevelopment 16
Refurbishment 18
New or enhanced Mobile 6
Table 3 below shows grants made by location type.
Table 3: Grants data by location type
Metro projects 22
Metro projects total allocation $8,131,000
Metro projects total allocation (% of total $) 47%
LLIP proportion of total projects’ expenditure, metro 6%
Regional projects 41
Regional projects total allocation $9,069,000
Regional projects total allocation (% of total $) 53%
LLIP proportion of total projects’ expenditure, regional
22%
Table 3 above shows that:
Total grant monies over the period 2011-2014 are roughly evenly divided between metro and regional library services.
Grants make up a larger proportion of total projects’ funding in regional library services compared to metro library services’ projects.
Figure 2 below shows that while library services in regional locations received almost twice as many grants by number, the majority were for smaller amounts, clustered under $100,000.
Living Libraries Infrastructure Program Evaluation Page 11 of 65
<$100,000
$100,000 to $199,999
$200,000 to $299,999
$300,000 to $399,999
$400,000 to $499,999
$500,000 to $599,999
$600,000 to $699,999
>$700,000
02468
101214161820
Comparison of Metro & Regional LLIP grants 2011-2014
METROREGIONAL
Size of grant
Num
ber o
f gra
nts
Figure 2: Comparison of Metro and Regional LLIP grants 2011-2014
Appropriateness and relevance
Key evaluation questions
The key evaluation questions for Appropriateness and Relevance of the LLIP are as follows:
What is the evidence of a continued need for the program?
What is the role for government in delivering this program?
To what extent does the Living Libraries Infrastructure Program continue to address a demonstrable need and is responsive to the needs of Victorians?
To what extent have options been investigated to address the identified need or problem?
Does the Living Libraries Infrastructure Program continue to be the best way to respond to the problem and deliver the intended outcomes?
How have the economic, environmental and social conditions changed since the program was funded and how will continuation of the program meet these conditions?
Findings
The Australian Library and Information Association (ALIA) articulates the need for public libraries in the following way (ALIA, 2009):
“Freedom of access to public library and information services is essential:
Living Libraries Infrastructure Program Evaluation Page 12 of 65
• to enable members of the Australian communities, including new residents and emerging communities to participate in all aspects of Australian life, including the democratic process;
• to actively contribute to social inclusion for all members of the Australian community; and
• to enable Australians to contribute to the economic wellbeing of their families and the nation.”
Public libraries in Victoria are entirely funded by government2, with the majority of funding, both operational and capital expenditure, by local government. The state government provides block funding for operational expenditures, and the LLIP for infrastructure capital expenditure. Library services may also be eligible for other state government grants, for example Community Support Grants.
Victorian public library services provided a total of 354 service points to the public, including 265 branches and thirty mobile services, according to the 2013-14 PLVN Performance Data report.
Significant research such as Libraries Building Communities (State Library of Victoria, 2005) and Dollars, Sense and Public Libraries (SGS Economics and Planning, 2011) has established the social and economic value of public libraries. Being The Best We Can (State Library of Victoria and Public Libraries Victoria Network, 2011), a libraries evaluation and improvement framework used by Victorian library services, summarised research as showing that libraries provide the following to their communities:
• provide gateways to information, learning and leisure;
• build individual skills, capability and wellbeing; and
• develop social connections and build social capital.
Successive governments have recognised the particular role of public libraries in fostering lifelong learning, and providing free access to information and a place for people to meet.
The LLIP is a response to identified needs around Victoria’s growth, changing community needs, the demands of ageing infrastructure, and a need to respond to modern developments in the library and information sector. Financial support is required to ensure that libraries’ infrastructure continues to meet Victorian communities’ diverse and growing needs,
The majority of Victorian public library services’ annual capital expenditure is provided by local governments: $29m of a total $33m in 2013-2014. The state government’s LLIP contributed an average $4.3m per annum to libraries’ infrastructure capital expenditure over the period 2011-2014. The LLIP funding made up 9.4% of the capital expenditure funding for projects by libraries over the period of the program. The proportion of the contribution made by LLIP to libraries’ projects is actually higher than this figure suggests, as many of the larger projects in the period were for multi-purpose buildings (e.g. Council ‘hubs’), so the library was only a portion of the total project expenditure.
2 With the exception of a small amount of funding raised through user charges and other sources.
Living Libraries Infrastructure Program Evaluation Page 13 of 65
The need for continuing investment in library services’ infrastructure is supported by interviewees and survey respondents in multiple comments about the value they have obtained from the LLIP and what would likely happen without it.
“Buildings have finite lifespans. The purpose for which they were built, and the changing nature of libraries, their buildings, and how they are being used and the purposes for which they are being used is changing and evolving. There is a need to recognise that by the state government. It is ‘public libraries’ still, despite what everyone says – it’s a partnership between local and state government I feel, and the contribution to the physical environment in which that happens is still very much a responsibility of both.” – library manager, metro Melbourne interface (high growth area).
The significance of the LLIP to the libraries’ infrastructure projects goes beyond the monetary value. Most library managers interviewed for the evaluation stated that without matching funding coming from the state government, their Councils would be much less inclined to fund the library projects.
“I think that you can’t underestimate just how critical the grants program is to actually getting commitment to capital funding. When you’ve got limited capital budget and you’ve got so many different services that are all trying to get that capital funding, when you can apply for a grant, it just adds so much weight to a capital bid to be able to have that with it …when it gets to executive or council if there’s a commitment of funding against it, they don’t want to knock back something where they’re potentially going to have co-funding.” – library manager, metro.
The continuing demand for, and responsiveness of the grant program is illustrated by the number of applications received over the period of the program. Of the forty-two respondents to the evaluation’s online survey, thirty-four (81%) had applied for a grant under the LLIP in the period 2011-2014. Thirty-three of these applicants were successful with one or more applications. Of the eight respondents who had not applied for an LLIP grant, five did not have infrastructure projects that would qualify under the LLIP, and one had an appropriate project that was already fully funded. One other respondent said that while they had not applied on behalf of their library service, three member councils of their regional library corporation had successfully done so.
Interviewees, survey respondents and research papers on Victorian public libraries reveal several other factors driving a continuing need for infrastructure investment:
Emergence of new technologies and evolution of established ones mean that buildings and library spaces need to change to meet the changing needs of ‘21 st
century literacies’.
Rapidly growing populations in metro Melbourne’s urban fringe regions are creating great demand for local services, including new and expanded public libraries.
The evolution of community use of local public spaces, especially public libraries, and their growth in use by the community as a safe ‘third space’ means that spaces in many long-established library buildings need to be reconfigured.
Living Libraries Infrastructure Program Evaluation Page 14 of 65
Effectiveness
Key evaluation questions
The key evaluation questions for Effectiveness of the LLIP are as follows:
What is the evidence of the program’s progress toward its stated objectives and expected outcomes, including the alignment between the program, its outputs, departmental objectives and any stated government priorities?
To what extent has Living Libraries Infrastructure Program progressed towards its stated objectives and outcomes it was seeking to achieve (at start-up and any revisions)?
Why was this program approach considered the best way to achieve the outcomes?
Findings
The LLIP’s long-term goal is to support the creation of strong, socially-inclusive, connected and active communities where people like to live, by helping Victorian public libraries to:
strengthen communities and encourage opportunities for community participation;
encourage and create lifelong learning opportunities for Victorian communities; and
facilitate free access to information and reading resources.The LLIP does this by providing grant funding to public libraries for infrastructure, comprising part of projects’ funding requirements. As a result, public library services are able to provide new or improved library infrastructure (construction works, permanent fixtures or fittings, or renovation works) that benefits their users and the community.The LLIP has contributed $17.2m in sixty-three grants, towards infrastructure works of over $180m over the period of 2011-2014.
According to the economic modelling in Dollars, Sense and Public Libraries, for every dollar invested in Victoria’s public libraries, the average value returned is $3.56 in community benefits.
Survey respondents rated the contribution made by the LLIP grant highly, as shown in the following figure.
Living Libraries Infrastructure Program Evaluation Page 15 of 65
Strongly disagree
Disagree Neither agree nor disagree
Agree Strongly agree0
5
10
15
20
25
The LLIP grant made a significant contribution to the infrastructure project
Num
ber
Figure 3: Contribution by LLIP grant
As described in the previous section, many interviewees and survey respondents stated that the LLIP grant was vital in ensuring the project went ahead in the first place.
In addition, many smaller projects, especially in regional areas, received a significant proportion of their total project funding from the LLIP. A number of these involved small upgrades and/or expansions to smaller branches.
“We would not be able to provide the library facilities required by our growing community without Living Libraries funding.” – survey respondent.
Large library projects have the potential to deliver very significant impact:
“Our old library, when it was on this site, used to have attendance figures or visitor figures of up to 15,000 people a month. At the moment, we’re running at 25,000 visitors a month, so there’s been a significant increase in the number of people visiting the facility. Our loans were of the order of 15,000 or 16,000 a month, they’re now between 23,000 and 30,000 a month so there’s been also a significant increase in the loans. In addition to that, other things like bookings of meeting facilities, we’re probably running at something like 300 bookings a month.” – library manager, metro interface (high growth) region.
Not only have projects made impact on usage, but also on user satisfaction in the services, as shown in the following figure.
Living Libraries Infrastructure Program Evaluation Page 16 of 65
Strongly disagree
Disagree Neither agree nor disagree
Agree Strongly agree0
5
10
15
20
25
A measurable increase in user satisfaction has resulted from the infrastructure project
Num
ber
Figure 4: User satisfaction impact
One example of the type of impact achievable by a relatively small amount of infrastructure funding was shown by surveys of user satisfaction before and after the redevelopment of a library branch in a small coastal town. The rating given by users for the range and quality of books increased from 9% to 67%, while the ease of locating a book went from 55% satisfaction to 100%. Overall user satisfaction was 100% (‘Satisfied’ or ‘Very satisfied’).
Strongly disagree
Disagree Neither agree nor disagree
Agree Strongly agree0
5
10
15
20
25
Measurable increases in usage have been made possible by the infrastructure project
Num
ber
Figure 5: Impact on usage
Both quantitative and qualitative effects on usage were described by interviewees and survey respondents.
Living Libraries Infrastructure Program Evaluation Page 17 of 65
“Massive [impact] is probably the first word that comes to mind. In the first day we had 120 people visit the library, we picked up about 33 new members.” – library manager, regional area, describing the effect of opening a new static branch replacing a visiting mobile service.
“We’ll be able to make the library a breastfeeding friendly venue, and be a place where mums can duck in and their older kids can read a book while they feed their babies.” – metro library manager describing new spaces made possible by the infrastructure investment.
Another example was given where the accessibility of services was increased significantly, with infrastructure grants making it possible for two renovated branches to increase their opening hours from fourteen hours per week to five and a half days.
Initiative delivery and efficiency
Key evaluation questions
The key evaluation questions for initiative delivery and efficiency are as follows:
To what extent has Living Libraries Infrastructure Program been delivered within its scope, budget and expected timeframe?
To what extent has Living Libraries Infrastructure Program been delivered in line with appropriate governance and risk management practices?
To what extent has Living Libraries Infrastructure Program been delivered at lowest possible cost without compromising quality?
What would be the impact of ceasing the program?
What would the impact be if DTPLI was to successfully exit from delivering Living Libraries Infrastructure Program if the government so desired?
What strategies have been identified to minimise any negative impacts of this exit?
If ongoing funding was provided, what level of efficiencies could be realised?
Findings
The LLIP has been delivered according to its scope, budget and planned timing – see Table 1 for details.
The cost of administering the LLIP is quite low, with one Equivalent Full Time (EFT) staff administering $17.2m of grants over four years, and dealing with forty-six public library services and other sector stakeholders. This resource is supplemented by convening of a part-time Grants Moderation Panel, an independent body that assesses applications against the grant guidelines and recommends successful applications for funding.
The Commonwealth Grant Guidelines, cited in Implementing Better Practice Grants Administration (Australian National Audit Office, 2009), provide a framework of principles that underpin better practice grant processes. The principles are:
Living Libraries Infrastructure Program Evaluation Page 18 of 65
1. Robust planning and design
2. An outcomes orientation
3. Proportionality
4. Collaboration and partnership
5. Governance and accountability
6. Probity and transparency
7. Achieving value with public money
Further details of the grants principles framework are shown in Attachment 7. Overall, the LLIP performs well against these principles, as demonstrated by the following findings.
Grants are decided in an annual process that starts with an open call for applications in February/March. The application process is relatively simple, using an Expression of Interest (EOI) format. The Department works with applicants to help them focus their project applications before they lodge them.
Interviewees and survey respondents alike rated the grants processes highly.
Local Government Victoria dealt with queries and other communications from us promptly
The grant funding was made available when we needed it
The timing of the call for applications fitted into our planning cycle
The grants were advertised far enough in advance to allow adequate preparation
The grant application process was easy to understand
1 2 3 4 5
Rating of the LLIP grants processes
Rating (scale 1 to 5)
Figure 6: Survey ratings of LLIP grants processes
Assistance from Local Government Victoria was highly appreciated by library service applicants.
“It [the grants process] was really good. I think mainly because we sat down with the group straight away, and they explained the process and what was needed, and I found that process really good and helpful.” – library manager, metro Melbourne interface (growth) area.
Living Libraries Infrastructure Program Evaluation Page 19 of 65
“It’s one of the simpler processes that I’ve been through.” – library manager, regional area.
Several interviewees and survey respondents did raise some issues around timing of the grants process compared to the annual planning processes of their Councils, which sometimes made it more difficult to get projects up in a timely manner.
“Formal advice of success was not received until July 1. Councils finalise their budget process in April each year – it would be better if a result was known prior to finalising the commitments of Council for each operating year.” – survey respondent.
Decisions on grant applications are made by the Grant Moderation Panel, which contains representatives of the Department, Public Libraries Victoria Network (PLVN), the Municipal Association of Victoria (MAV) and the State Library of Victoria (SLV). Interviewees and survey respondents were satisfied that the grant requirements about the scope and type of projects eligible for funding were adequate and reasonable, and were clearly expressed in the published grant guidelines. However, several comments were made that questioned the consistency of decisions made.
A significant proportion of interviewees and survey respondents commented that the size of the maximum grant could reasonably be increased beyond $750,000 given the escalation in building costs for new libraries. This is a significant issue for larger interface Councils and Regional Library Corporations that are facing rapid population growth and are expected by the community to respond with major new buildings. Even though the LLIP contribution to such large projects may be a relatively small proportion of the whole, it can make a significant difference to the overall quality of the finished product and also to a Council’s willingness to proceed, according to a number of those interviewed for the evaluation.
Many interviewees commented that smaller grants often make very significant impacts on smaller library services, especially in regional areas. Grants even of under $100,000 can make large differences to the quality of users’ experience in smaller branches or mobile services.
Many interviewees and survey respondents commented that without the LLIP grant component, their project would not have proceeded. A common reason for this is that, on the one hand, the Council would not consider proceeding unless there was a matching component form the state government, and on the other hand that once funding was approved from the state government, it acted as a strong incentive to proceed, because Council did not want to forego promised funds. These comments were reinforced by those made by unsuccessful applicants, more than three-quarters of whom said their project did not proceed because they failed to win the LLIP funds.
All those interviewed and surveyed for the evaluation agree that were the LLIP to be terminated without replacement, the consequences for public libraries’ infrastructure in Victoria would be significantly negative. Political fallout could be expected as a result, according to a number of interviewees.
Living Libraries Infrastructure Program Evaluation Page 20 of 65
Lessons and future directions
Lessons on initiative design and delivery
Overall, the design appears appropriate and adequate for the purpose expressed in the program objectives. Applications run at about double the amount of funding available each year, so there is sufficient competition between grantees to ensure that good applications proceed. Unsuccessful applications are often fine-tuned between grant calls, resulting in better submissions that are more likely to succeed the following year. The Department plays an appropriate role in ensuring applicants put their best proposal forward.
A wide variety of comments were made by interviewees and survey respondents about potential improvements to the LLIP. The main themes were to:
Increase the maximum grant size (some suggested a figure of $1m).
Widen the scope of applicable infrastructure to cover a defined range of furnishings (restrictive definitions of fixed infrastructure can limit libraries’ ability to respond to changing usage needs).
Many of those interviewed and surveyed commented on the wide range of projects funded under the LLIP. This was very much seen as a positive, because no-one was excluded. The differing levels of matching funding on offer were widely commented upon as being appropriate, as they reflected differing levels of need and financial circumstances between library services in metropolitan Melbourne, regional cities, metro interface areas, and regional and rural councils.
Impact of stopping the initiative
It is unlikely the Department could successfully stop delivering the initiative without there being significant backlash from the public library sector, local government and the community, potentially leading to political fallout. The state government is seen to be responsible for providing public library services in collaboration with local government, and cessation of the LLIP could be interpreted as withdrawal of the state government from its publicly stated commitment to supporting and enhancing the quality of public library services in Victoria.
Living Libraries Infrastructure Program Evaluation Page 21 of 65
Conclusions and recommendations
Conclusions
The Living Libraries Infrastructure Program is a grant funding initiative that is seen by Victorian public library services as a vital and integral part of the sector’s annual infrastructure funding. It contributes around 10% on average of public libraries’ infrastructure spending and is considered to play a vital role in locking in counterpart funding from local Councils.
Grant processes are viewed as efficient, open and transparent, simple to comply with for applicants, and delivered in a collaborative manner between the Department and applicant public library services.
Recommendations
It is recommended that:
1. The Department review the maximum grant size.
2. The Department review the rationale for deciding the limitations to non-fixed infrastructure for grant funding.
3. The Department review whether there is a case for articulating decision making guidelines on a policy basis for annual division of the grant funds pool between large, medium and small projects.
4. The Department articulate the grant decision-making principles in the guidelines made available to applicants.
5. The Department consider changing the timing of the grant process to better fit library services’ capital budget planning processes, for example by running multiple grant rounds each year or a continuously open round. There may be merit in running separate processes for large and small projects.
Living Libraries Infrastructure Program Evaluation Page 22 of 65
Attachments
Attachment 1: Evaluation design
The evaluation design detailed below was designed to meet DTF requirements, including recommendations for future program delivery improvements, based on evidence gathered in the evaluation.
Purpose of the evaluation.
The purpose of the evaluation is to provide information on the effectiveness of the Living Libraries Infrastructure Program and determine whether the program has met the appropriate policy objectives, planned outputs and desired outcomes. The evaluation will also inform the government's current and future policy and decision-making around the program – a new business case will be developed early in 2015 for a proposed new round of the program.
Scope of the evaluation.
The evaluation will comprise:
1. Collection and analysis of data relating to library services that have received funding as part of the Living Libraries Infrastructure Program;
2. Analysis of relevant reports and surveys of library professionals; and
3. A review of relevant documents.
The main quantitative data collected will be derived from the 2013–14 public library statistics compiled by PLVN on behalf of Local Government Victoria. Additional data may be sourced from specific library services via formal surveys to help ascertain the benefit of facilities built as part of the living libraries infrastructure program. Qualitative data will be obtained through evaluation research, and LGV will contribute case study information from a representative sample of eight libraries from around the state.
Target audiences.
The main audience for the evaluation is Local Government Victoria, which will use the evaluation as part of its budget submission preparation.
Key evaluation questions.
The evaluation will assess the program against the requirements of the Department of Treasury and Finance’s Evaluation Policy and Standards for Lapsing Programs:
1. justification of the program and links to policy;
2. effectiveness;
3. funding/delivery;
4. risk; and
5. efficiency.
Living Libraries Infrastructure Program Evaluation Page 23 of 65
The detailed evaluation questions to be answered are as follows:
1. Justification / problem: What is the evidence of a continued need for the program? What is the role for government in delivering this program?
To what extent does the Living Libraries Infrastructure Program continue to address a demonstrable need and is responsive to the needs of Victorians?
To what extent have options been investigated to address the identified need or problem?
Does the Living Libraries Infrastructure Program continue to be the best way to respond to the problem and deliver the intended outcomes?
How have the economic, environmental and social conditions changed since the program was funded and how will continuation of the program meet these conditions?
2. Effectiveness: What is the evidence of the program’s progress toward its stated objectives and expected outcomes, including the alignment between program, its outputs (as outlined in Budget Paper 3), departmental objectives and any stated government priorities?
To what extent has Living Libraries Infrastructure Program progressed towards its stated objectives and outcomes it was seeking to achieve (at start-up and any revisions)?
Why was this program approach considered the best way to achieve the outcomes?
3. Funding / delivery: (a) Has the program been delivered within its scope, budget, within expected timeframes, and in line with appropriate governance and risk management practices; and b) Has the department demonstrated efficiency and economy in relation to the delivery of the program?
To what extent has Living Libraries Infrastructure Program been delivered within its scope?
To what extent has Living Libraries Infrastructure Program being delivered within its budget?
To what extent has Living Libraries Infrastructure Program been delivered within the expected timeframe?
To what extent has Living Libraries Infrastructure Program been delivered in line with appropriate governance and risk management practices?
To what extent has Living Libraries Infrastructure Program been delivered at lowest possible cost without compromising quality?
4. Risk: What would be the impact of ceasing the program?
What would the impact be if DTPLI was to successfully exit from delivering Living Libraries Infrastructure Program if the government so desired?
Living Libraries Infrastructure Program Evaluation Page 24 of 65
What strategies have been identified to minimise any negative impacts of this exit?
5. Efficiency: What efficiencies could be realised?
If ongoing funding was provided, what level of efficiencies could be realised?
Logic model.
The program outcomes logic model for the Living Libraries Infrastructure Program is shown in Attachment 2.
Interview guide and survey questionnaire.
The interview guide is shown in Attachment 3; the survey questionnaire is shown in Attachment 4.
Evaluation plan and timetable
The evaluation plan timetable is shown in Figure 7 below, and explained in the following paragraphs.
Week CommencingDec Dec Dec Dec Dec Jan Jan Jan Jan Feb
Outputs 01 08 15 22 29 05 12 19 26 02
1. PLANCommencement meeting Contract & draft workplan
Review documentationDevelop program logic model PLM
Stakeholder analysis Stakeholder engagement plan
Develop evaluation framework & questions Framework & KEQs
Develop interview & survey questions & tools Questionnaires, interview guides
Finalise methodology and evaluation plan Evaluation Plan
2. COLLECT DATAConduct stakeholder interviews Qual. data
Collect & review library data Quant. data
Conduct online survey Survey data
3. ANALYSE & REPORTAnalyse quantitative & qualitative data Thematic analysis
Assess improvement opportunities Recommendations
Draft evaluation report Draft evaluation report
Circulate for review Revisions requested
Finalise report Final evaluation report
PHASE & Tasks
1
3
2
= milestone deliverable
Figure 7: Evaluation timetable
Commencement meeting
At the commencement meeting we will confirm the evaluation scope as being to determine the effectiveness, efficiency and equity of program delivery and to assess achievements against intended program objectives and outcomes.
Living Libraries Infrastructure Program Evaluation Page 25 of 65
The evaluation will consider:
Program justification: the need that the program addresses;
Program effectiveness: the level of achievement of the program outcomes, including an assessment of success factors and constraints and linkages to departmental objectives and stated government priorities;
Funding/delivery and efficiency: program management and processes;
Risk: impact of ceasing the program; and
Efficiencies realisable in a continuing program.
Review documentation
At the commencement meeting, or immediately following it, LGV will provide as much of the relevant program documentation as is practicably available for us to review.
Develop program logic model
As this evaluation examines outcomes and also seeks to discover program improvement possibilities, we will start by articulating in summary form the program design and implementation steps and the objectives they were intended to achieve.
Working from program documentation and discussions with LGV, we will formulate an overview of the program logic, articulating how resources were intended to be mobilised, what processes were used for implementation, what outcomes were expected, and what longer-term outcomes are anticipated.
Stakeholder analysis
In consultation with the project manager, ACIG will develop a list of stakeholders to be consulted and develop a consultation plan.
We expect that we will consult selected stakeholders through an appropriate mix of face to face and telephone interviews.
Develop evaluation framework & key questions
Having confirmed the overall program logic, we will use it to inform the evaluation framework.
To frame the evaluation, we will firstly confirm in consultation with the evaluation project manager the set of questions that the evaluation is to answer. The top level questions and sub-questions have been provided in the brief, based on DTF requirements.
The detailed list of evaluation questions will form the basis for data collection tools such as interview plans and surveys.
Living Libraries Infrastructure Program Evaluation Page 26 of 65
Develop interview questions & tools
To ensure consistency we will develop an interview plan that lists key discussion points to be covered during the course of the interview, without circumscribing the conversation. We will use the interview plan as a prompt, then probe the interviewee for further depth or follow up on issues raised.
Finalise methodology and evaluation plan
We will then finalise the detail of the evaluation plan, working from the agreed framework. The evaluation plan will include:
1. Purpose of the evaluation.
2. Scope of the evaluation.
3. Target audiences.
4. Key evaluation questions.
5. Logic model – diagram and narrative.
6. Evaluation indicators – both quantitative and qualitative.
7. Methods for key informant engagement and data collection, aggregation and analysis.
8. Interview guides.
Collect Data
Both quantitative and qualitative data will be collected for analysis in the evaluation. Our primary sources of evidence will be: quantitative data provided by LGV; documents provided by LGV; survey data; and key informant/stakeholder interviews. We will create a data collection matrix that will serve both as a data collection guide and as cross-referencing summary table for the initial analysis.
Quantitative data analysis
We will analyse data provided by LGV that may include (but is not limited to):
Program funding and resourcing;
Data evidencing before-project and after-project situations, to enable assessment of change attributable to a sample of projects funded by the Living Libraries Infrastructure Program grant, for example:
- Spaces (e.g. amount, type, availability for specific usages)
- Collection stock (e.g. type, quantities, average age)
- Membership numbers
- Loans
- Availability and accessibility data
Living Libraries Infrastructure Program Evaluation Page 27 of 65
Qualitative data analysis
The following qualitative data will also be analysed:
Existing qualitative feedback received from grant applicants;
Review of grant project outcomes;
Information from surveys of and interviews with grant applicants and LGV program officers;
Information on the Living Libraries Infrastructure Program grants process, including program guidelines and evidence of the implementation of the process; and
Examine the most efficient program delivery mechanism (i.e. the market place or government) where possible.
Online survey
We propose developing a brief online survey to ascertain the benefit of facilities built as part of the Living Libraries Infrastructure Program. The survey will be sent directly to grant recipients. We have over 20 years of experience in survey design and analysis, so we know how to structure surveys in order to obtain clear and practical results.
Key informant interviews
Based on our understanding of the brief, we anticipate up to twelve key informant/stakeholder interviews, depending on the final stakeholder analysis. Although the Terms of Reference do not explicitly mention interviews, we regard interviews with key stakeholders an essential part of the data collection, for triangulation purposes.
We will seek LGV’s assistance in initially contacting interviewees and validating our interview request. Most interviews will be less than one hour long, and we expect to be able to conduct most (if not all) of them by telephone.
Document review
We will carefully examine the program documentation provided to seek evidence about processes that were implemented, and outcomes that have been achieved through those processes. These data will be cross-referenced in the data collection matrix. We will include review of information on best practice government grants administration processes such as the ANAO guidelines.
Preliminary analysis
Having tabulated the relevant evidence in the data collection matrix, we will examine each evidence cluster to assess the extent of achievement of intended outcomes to date. We will review the documentation for evidence of unintended outcomes (positive or negative) and, if necessary, follow up with enquiries for clarification.
Living Libraries Infrastructure Program Evaluation Page 28 of 65
Assess improvement opportunities
We will review the complete data set – quantitative and qualitative – to develop a list of potential program improvements. We will consult further with LGV to refine those opportunities for improvement, as it is important at this time that we leverage the in-depth program knowledge held by department staff.
Draft report
We assume that the audience for this evaluation report is internal to government. The report is to meet DTF’s Evaluation Policy and Standards for Lapsing Programs and (as specified by DTF) shall have no more than thirty pages in the body. Our findings will be documented in this report, which will address the core lapsing program questions.
The draft report will include the following sections.
Executive Summary
Introduction
Findings
Conclusions and recommendations
We will present a final draft of the report to the evaluation project manager for any final comments, before circulating it for wider review.
Circulate report for review
We will rely on DTPLI/LGV to ensure the review is completed and comments and revisions returned to us in time for the report to be finalised according to the department’s timetable.
Finalise report
Following the report review period we will finalise the report.
Living Libraries Infrastructure Program Evaluation Page 29 of 65
Attachment 2: Outcomes logic modelStrategies
Encourage investment in new and improved public library infrastructure by part-funding projects by library services with grants up to $750,000
Goal
Provide support by renewing public library infrastructure and addressing gaps in service provision.
Policy
The Victorian Government recognises the particular role of public libraries in fostering lifelong learning, and providing free access to information and a place for people to meet.
Problem
Victoria is growing, and support is required to ensure that libraries continue to meet Victorian communities’ diverse and growing needs, and respond to the demands of ageing infrastructure, changing community needs, and modern developments in the sector.
Inputs
Government funding.
Activities
Annual infrastructure grants program.
Outputs
Grant funding provided to public libraries for infrastructure, comprising part of projects’ funding requirements.
Clients
Potentially – all fifty-two Victorian public library services.
Key stakeholders
Victorian public library services.
Local Government Victoria
Public Libraries Victoria Network
Municipal Association of Victoria
OutcomesShort-term
Public library services provide new or improved library infrastructure (construction works, permanent fixtures or fittings; or renovation works).
OutcomesMedium-term
Victorian public library facilities improved.
Regional libraries and local governments provide high-quality and accessible public library infrastructure.
OutcomesLong-term (Impact)
Public libraries supported in:
strengthening communities and encouraging opportunities for community participation;
encouraging and creating lifelong learning opportunities for Victorian communities; and
facilitating free access to information and reading resources.
Strong socially-inclusive, connected and active communities where people like to live.
Living Libraries Infrastructure Program Evaluation Page 30 of 65
Attachment 3: Interview guide
Objective
Grantee interviews for the evaluation explore the following key evaluation questions: To what extent has the program met stakeholders’ expectations? What outcomes has the program achieved? What would happen if the program (or similar infrastructure upgrade/replacement
funding process) did not exist? Is there a continuing need for the program and does LGV have a continuing role to
play in improving public libraries’ infrastructure?
Introduction
Thank the interviewee for giving the time for this interview. Ask permission to record the interview, to enable accurate note-keeping.
Local Government Victoria has engaged the Australian Continuous Improvement Group (ACIG) to undertake an evaluation of the Living Libraries Infrastructure Program. ACIG is an independent management consultancy engaged in evaluation, benchmarking and continuous improvement for clients for more than twenty-two years.
This independent evaluation will be conducted within the Department of Treasury and Finance guidelines and will be used to inform the development of the 2015 budget funding submission.
Data collection for the evaluation includes document review, surveys and interviews. You have been chosen as part of a random sample of the 63 projects funded over the past four years of the program. We would like to ask you some questions about the program and your experience of it.
Please feel free to be open and frank in their answers. All information will be de-identified in our report to LGV.
(Be prepared to answer questions about what access there will be to the final evaluation report.)
Opening
Confirm the interviewee’s role and the type and scope of their involvement with the program: grants applied for; grants won; projects completed and in-progress.
Grant application(s)
What has your experience been with the grants process?
Were there any problems?
How could the process be improved?
Project delivery
Was funding made available when it was needed? As planned?
Were any obstacles encountered? How were they dealt with?
What could have been done better? In what way?
Living Libraries Infrastructure Program Evaluation Page 31 of 65
How well do you feel the LGV program office communicated with stakeholders?
Were there any issues? What caused them? What could have been done better?
Outcomes
What outcomes were achieved as a result of the grant funding?
Do you have data on improved outcomes, e.g. increased usage of library services, changes in community satisfaction, increased access to resources, etc.?
How significant to the project outcomes was the LLIP program funding received?
What would have happened if you had not received the LLIP funding for the project? Would the project have still proceeded? What changes would have been necessary?
Do you have any comments about:
the size of the grants offered, the conditions of the grant, or what types of library services are or should be eligible for them?
Do you feel there is a continuing need for this type of grant funding for library infrastructure from the State Government?
If the program ceased, what would be the impact?
Closing
Thank the interviewee once again for their time. Leave them with contact details, should they have any further comments or questions.
Living Libraries Infrastructure Program Evaluation Page 32 of 65
Attachment 4: Survey instrument
Living Libraries Infrastructure Program Evaluation Page 33 of 65
Living Libraries Infrastructure Program Evaluation Page 34 of 65
Living Libraries Infrastructure Program Evaluation Page 35 of 65
Living Libraries Infrastructure Program Evaluation Page 36 of 65
Living Libraries Infrastructure Program Evaluation Page 37 of 65
Living Libraries Infrastructure Program Evaluation Page 38 of 65
Attachment 5: Survey data
Living Libraries Infrastructure Program Evaluation Page 39 of 65
Living Libraries Infrastructure Program Evaluation Page 40 of 65
Living Libraries Infrastructure Program Evaluation Page 41 of 65
Living Libraries Infrastructure Program Evaluation Page 42 of 65
Living Libraries Infrastructure Program Evaluation Page 43 of 65
Living Libraries Infrastructure Program Evaluation Page 44 of 65
Living Libraries Infrastructure Program Evaluation Page 45 of 65
Living Libraries Infrastructure Program Evaluation Page 46 of 65
Living Libraries Infrastructure Program Evaluation Page 47 of 65
Living Libraries Infrastructure Program Evaluation Page 48 of 65
Living Libraries Infrastructure Program Evaluation Page 49 of 65
Living Libraries Infrastructure Program Evaluation Page 50 of 65
Living Libraries Infrastructure Program Evaluation Page 51 of 65
Living Libraries Infrastructure Program Evaluation Page 52 of 65
Living Libraries Infrastructure Program Evaluation Page 53 of 65
Living Libraries Infrastructure Program Evaluation Page 54 of 65
Living Libraries Infrastructure Program Evaluation Page 55 of 65
Living Libraries Infrastructure Program Evaluation Page 56 of 65
Living Libraries Infrastructure Program Evaluation Page 57 of 65
Living Libraries Infrastructure Program Evaluation Page 58 of 65
Living Libraries Infrastructure Program Evaluation Page 59 of 65
Living Libraries Infrastructure Program Evaluation Page 60 of 65
Living Libraries Infrastructure Program Evaluation Page 61 of 65
Attachment 6: References
Australian Library and Information Association. 2009. Statement on public library services. Canberra.
Australian National Audit Office. 2010. Implementing Better Practice Grants Administration. Commonwealth of Australia. Canberra.
SGS Economics and Planning. 2011. Dollars, Sense and Public Libraries. State Library of Victoria. Melbourne.
State Library of Victoria. 2005. Libraries Building Communities – The Vital Contribution of Victoria’s Public Libraries – A Research Report for the Library Board of Victoria and the Victorian Public Library Network. Melbourne.
State Library of Victoria and Public Libraries Victoria Network. 2011. Being The Best We Can – Key Results for Public Library Services. Melbourne
Living Libraries Infrastructure Program Evaluation Page 62 of 65
Attachment 7: Australian Library and Information Association Statement on Public Library Services
Principle
Freedom of access to public library and information services is essential: to enable members of the Australian communities, including new residents and
emerging communities to participate in all aspects of Australian life, including the democratic process;
to actively contribute to social inclusion for all members of the Australian community; and
to enable Australians to contribute to the economic wellbeing of their families and the nation.
Statement
Each member of the Australian community has an equal right to public library and information services regardless of age, race, gender, religion, nationality, language, disability, geographic location, social status, economic status and educational attainment.
A public library services its community through the provision of access to knowledge, information and works of imagination through a range of resources and services. It does this through access to materials in all formats in order to meet the needs of individuals and groups for education, information and personal development through learning, including recreation and leisure.
Public libraries have an important role in the development and maintenance of a democratic society by giving individuals access to a wide and varied range of information, ideas, opinions, and skills.
The role of public libraries is essential in developing an educated society through programs that improve literacy and information literacy including lifelong learning opportunities. Public libraries contribute to economic prosperity by helping people improve their skills and life chances.
Public libraries provide the first point of access for information for the general public and for the public's access to the national and international system of library and information services.
The satisfaction of a person's information needs must be independent of an ability to pay.
Local, state/territory and Commonwealth governments have an obligation to work in partnership to provide agreed public library services to all members of the library's clientele without direct charge to the user.
The Australian Library and Information Association believes that public library services have particular responsibilities to monitor and proactively respond to the changing demographic characteristics and trends of their communities, to consult with their communities and to meet information, learning and recreational needs of an increasingly diverse society. Public library services should ensure that they have policies, guidelines, and procedures in place to respond to and meet relevant legislative requirements.
Living Libraries Infrastructure Program Evaluation Page 63 of 65
Attachment 8: Australian National Audit Office Principles for Better Practice Grant Processes
The Commonwealth Grant Guidelines, cited in Implementing Better Practice Grants Administration (Australian National Audit Office, 2009), provide a framework of principles that underpin better practice grant processes.
The principles are:
1. Robust planning and design which underpins efficient, effective and ethical grants administration, including through the establishment of effective risk management processes.
2. An outcomes orientation in which grants administration focuses on maximising the achievement of intended government outcomes from the available funding.
3. Proportionality in which key program design features and related administrative processes are commensurate with the scale, nature, complexity and risks involved in the granting activity.
4. Collaboration and partnership in which effective consultation and a constructive and cooperative relationship between the administering agency, grant recipients and other relevant stakeholders contribute to achieving more efficient, effective and equitable grants administration.
5. Governance and accountability in which a robust governance framework is established that clearly defies the roles and responsibilities of all relevant parties; establishes the policies, procedures and guidelines necessary for defensible funding recipient selection and administration processes that comply with all relevant legal and policy requirements; and supports public accountability for decision-making, grant administration and performance monitoring.
6. Probity and transparency in which program administration reflects ethical behaviour, in line with public sector values and duties; incorporates appropriate internal and fraud control measures; ensures that decisions relating to granting activity are impartial, appropriately documented and publicly defensible; and complies with public reporting requirements.
7. Achieving value with public money which should be a prime consideration in all aspects of grant administration and involves the careful consideration of costs, benefits, options and risks.
Living Libraries Infrastructure Program Evaluation Page 64 of 64