76
1 Understanding and Implementing the New Annual Performance Report (APR)
1
Understanding andImplementing the New Annual
Performance Report (APR)
2
Presenters
• Natalie Matthews, Abt Associates• Louise Rothschild, Abt Associates• Darlene Mathews, Darlene Mathews, Inc.
3
Learning Objectives1. Understand the content, requirements and transition
to the new Annual Performance Report.2. Review the specific content of the HPRP APR.3. Understand what e-snaps is, and how to use it to
submit the new APR.4. Identify opportunities for new uses of the APR in your
program and community (including performancemeasurement, data quality monitoring, and fundingdecisions).
4
Annual Performance Report: Overview
5
Goals of the APR
1. Used by HUD to inform the Department’s competitiveprocess for homeless assistance funding and itprovides a vehicle for looking at program compliance
2. Used by grantees to report to the HUD on programaccomplishments and outcomes over the course ofan operating year
6
New APR: What’s Changed?– Electronic, web based submittal via e-snaps– e-snaps will allow you to only see those questions that relate to
your grant and component type (example: SHP-SSO)– Is aligned with the HMIS Data Standards (released March 2010)– Reports information on “leavers” and “stayers”– New types of households– No longer breaks out data for clients who are “Chronically
Homeless”– Reports on performance measures set by the project during the
grant application process– Allows client progress to be tracked over time
7
Objectives for New APR
• Part of HUD’s overall effort to streamline and integratethe grant application, management, and reportingprocesses
• Changes will allow for grantees to more accuratelyand comprehensively report on what occurred duringthe operating year
• New performance measures will provide anopportunity to quantify what grantees accomplished
8
Adults and Children
• Most information on the APR is reported on bothadults and children
• Adult: Any person 18 years of age or older.• Children: Any person under the age of 18.
9
Key Definitions: Households
1. Households without Children: Includes unaccompaniedadults, multiple adult households, and pregnant women notaccompanied by other children.
2. Households with Children: Any household with at least oneadult and one child.
3. Households with only children-Contains one or more personsage 17 and under including unaccompanied youth, adolescentparents and their children and adolescent siblings
10
Key Definitions: Leavers & Stayers
• Leaver: Clients who exited and were not in theprogram on the last day of the operating year
• Stayer: Clients who were in the program on the lastday of the operating year. Includes clients who exitedthe program and re-entered the program before theend of the operating year.
11
Audience Quiz 1: How would you report ahousehold consisting of a 17-year oldpregnant female and her 15 year oldbrother?1. As a household without children2. As a household with one adult and one
child3. As a household with only children4. As a family
12
New APR Format
Seven sections to the new APR• Project Information• Program Outputs• Client Characteristics• Financial Information• Program Performance• Narrative• HMIS Dedicated Projects
13
1. Project Information• Draws heavily from information provided by grantee in the grant
application and/or technical submission• Critical that the information reported on the APR in this section,
matches what was reported on the application for funding or yourmost recent technical submission
• Sample questions:– Project information– Authorizing Information– Bed and unit inventory– HMIS bed participation rate and HMIS Data Quality
14
Project Information
15
2. Program Outputs
• Report out on the volume of work accomplished overthe course of the operating year
• Relies heavily on data collected in HMIS, as well asitems already reported out in the Housing InventoryChart
• Sample questions:– Number of persons and households served– Bed and unit utilization rates– Number of client contacts (street outreach programs)
16
Program Outputs
17
3. Client Information
• Report out on the demographics of who was served ina program, as well as the progress a client makes overthe course of an operating year
• Data is now broken out by “household type” for severalquestions
• Sample questions:– Age, gender, race, ethnicity, veterans status,– Income and Non-Cash benefits (changes from program entry
to program exit)– Physical and Mental Health Conditions– Destination by Household Type and Length of Stay
18
Client Monthly Cash-Income Amount
19
Exit Destination
20
4. Financial Information
• Reports out on information about project expendituresand matching amount
• Questions are separated out by grant type• Sample questions:
– S+C and Supportive Services Match Expenditures– Financial Information for SHP– Financial Information for HPRP
21
5. Program Performance• Report out on performance measures by program type• Grantees will define program measures during the
application for funding process• Sample questions:
– Permanent housing program performance measure: The % ofpersons who remained in the permanent housing program as of theend of the operating year or exited to permanent housing (subsidizedor unsubsidized)
– Transitional housing programs performance measure: The % ofpersons age 18 or older who increased their total income (from allsources) as of the end of the operating year or program exit
22
6. Narrative
• Provides an opportunity for programs to providedescriptive information to HUD
• Sample questions:– Explanation of variances between planned and actual
performance– Additional comments
23
7. HMIS Dedicated Projects• Report out on basic information about the scope of the
HMIS implementation, HMIS functionalities, HMISprogram and bed coverage, training, and data quality.
• Much of this information is already reported on theCoC’s Exhibit 1
• Sample questions:– HMIS participation by program type– HMIS functionality (general, HUD reporting, data quality,
security, data exchange)– User training type, requirements, frequency, and completion– Funding and expenditures
24
HMIS Data Quality
25
Audience Quiz 2: What is a key differencebetween the HUD-40118 APR and the newAPR?
1. Nothing, they are identical2. The new APR needs to be submitted
electronically in e-snaps3. The new APR reports information on
both leavers and stayers4. Both 2 and 3
26
Data Collection• Data collection with the final revised HMIS
Data Standards began on June 1, 2010 forboth CoC projects and HPRP grantees.
• CoC projects (SHP, S+C, SRO) will need onefull year of data collection with the final revisedHMIS Data Standards (released March, 2010),before they will be able to fully complete thenew APR.
27
Data Collection (continued)
• Programs may need to back enter data toensure that clients served on or after June 1,2010 to have complete records. This will allowgrantees to submit full APRs in e-snaps foroperating years ending June 1, 2011 or after.
28
Transition to the new APR: ReportingRequirements for CoC Projects
New APRHUD-40118 APR
Transition APR
29
Transition APR: Submission Requirements
• Submitted in e-snaps• Required for CoC projects with operating
years ending between July 1, 2010 and May31, 2011
• Due 90 days from the end of the operatingyear
30
Transition APR: Submission Requirements(cont’d)• If your vendor programmed an update to the HUD-
40118 so that it can still be run, then use theTransition APR Guidebook and resources.
• If you vendor programmed the new APR, then you willjust match your APR to the e-snaps screens andsubmit the answers to all fields that appear on yourscreen.
TALK TO YOUR HMIS VENDOR!
31
Transition APR-What’s included
• Includes only those questions that:– Were also on the HUD-40118 APR– Program Information that should be readily
available (Sample question: Project Information,Site Address)
– Some questions that were in the 40118 APR withexpanded answer choices
32
Transition APR: Resources• Transition APR Guidebook
– Maps every question from the HUD-40118 APR to theTransition APR
– Step by step instructions on how to complete the TransitionAPR
• Transition APR Training– Online, 30 minute tutorial that reviews all of the requirements
for the Transition APR, and demonstrates how to answerseveral questions using data from the HUD-40118 form
– Can be downloaded at any time, and paused if you need tostop and revisit
33
APR: Key Dates
Grant type Operating Year enddate
APR Requirements
CoC Projects (SHP, S+C,SRO)
On or before June 30,2010
HUD-40118 APR form
CoC Projects (SHP, S+C,SRO) *NOT includingHMIS dedicated projects
On or after July 1, 2010and before May 31, 2011
Transition APR in e-snaps
HMIS dedicated projectsonly
On or after July 1, 2010and before May 31, 2011
New APR in e-snaps, withthe exception of dataquality questions
CoC Projects (SHP, S+C,SRO)
On or after June 1, 2011 New APR in e-snaps
34
Audience Quiz 3: Who needs to completea Transition APR?
1. SHP, S+C, and SRO grantees with operating yearsending between June 1, 2010 and May 31, 2011
2. SHP, S+C, and SRO grantees with operating yearsending between July 1, 2010 and May 31, 2011
3. All HUD grantees, regardless of grant type andoperating year end date
4. Only grantees that irritate their local field office
35
e-snaps
36
e-snaps demo
• Setting up an APR• Tips for using e-snaps
www.HUDHRE.info/apr
Grantees should use the APR Guidebooks for step-by-stepinstructions on how to use e-snaps for the APR. TheTransition APR and HPRP APR online trainings alsoinclude demonstrations of how to use the system
37
38
39
40
41
42
43
44
45
46
47
Audience Quiz 4: How do I access myAPR in e-snaps?
1. Log in to e-snaps, and the APR willinstantly appear on the screen
2. Request access from HUD3. The process is unique depending on the
grant type and operating year end date.Follow the detailed instructions in the APRGuidebook for your APR type (Transition,HPRP, SHP-HMIS or CoC).
4. Who knows? We haven’t covered that.
48
49
50
51
Key Tips for Using e-snaps
• Responses to question 3 are critical to determiningwhat questions you will see on the APR
• Use the Guidebooks on HRE for step-by-stepinstructions on completing the APR in e-snaps
• Always remember to hit “Save!”• Don’t share log in information with anyone• System can run slow if many people are using it at the
same time, so don’t wait until the last minute to submit
52
HPRP APR
53
HPRP APR: Reporting Requirements
• All HPRP APRs are due by November 30th
• Reporting period covers October 1st-September 30th
– HPRP Programs that began prior to October 1, 2009 shouldreport all data from the date that they began operating,through September 30th, 2010 in the first APR, due no laterthan November 30th, 2010.
• Must be submitted in e-snaps by the HPRP Grantee(same process that is used for the QPR)
• Grantees must use HMIS or comparable client-leveldatabase to produce performance data for the QPR
54
HPRP APR (continued)
• An HPRP grantee must complete only one APR foreach grant.
• Like the QPR, the HPRP APR must aggregate andunduplicate information for all subgrantees to reportinformation in the HPRP APR for the grantee
• Information broken out by Prevention and HomelessAssistance Services
55
HPRP APR- Data Collection
• Collecting all necessary client level information inHMIS (or a comparable database) since HPRP began,using the June 2009 draft revised HMIS DataStandards
• All HPRP projects should now be collecting data underthe final revised HMIS Data Standards that werereleased in March 2010
• As of June 1, 2010 ALL client data should be collectedaccording to those new standards—including anyupdates made to current client records
56
HPRP: Persons Enrolled
• Persons Enrolled question counts how many personswere served in HPRP Programs
• In order for a client to be counted as Person Enrolledthey must have a service recorded in the HMIS duringtheir stay.
• An HPRP service activity is any financial assistance orhousing relocation and stabilization service that isfunded through HPRP.
57
HPRP APR- Key Points
• Much more information is reported out in the APR,than what is currently reported in the QPR
• Designed to look at clients by:– Homelessness Prevention and Homeless Assistance (Rapid
Re-Housing) services– Housing Status at Entry/Exit– Length of stay in a program
58
HPRP APR- Questions
• Income and non-cash benefit amounts, sources andchanges over program participation (leavers only)
• Length of Participation by Homelessness Preventionand Homeless Assistance (leavers only)
• Destination for leavers staying 90 days or more• Financial Assistance• HPRP Performance Measures• Project description
59
HPRP APR: Questions (continued)
• Project information• Data quality of HMIS and Comparable Database• Persons/households enrolled• Client Characteristics:
– Age– Gender– Race– Ethnicity– Residence Prior to Program Entry– Veteran status
60
Audience Quiz 5: When is someonereported in the HPRP APR as enrolled?
1. When s/he is screened for eligibility for HPRP2. When the client’s case manager decides they can
receive services3. When they have an HPRP program entry date and
have received an HPRP service4. When they are stably housed
61
Let’s now review some of these questions inmore detail…
62
HPRP APR: Resources and Tools
• HUD HRE:– HPRP APR Guidebook is available on HUD HRE to walk
grantees through the process– HPRP APR Training that provides an overview of the content
in the HPRP APR– FAQ’s that are updated regularly– APR Programming Specifications, version 2.0 (specific to the
APR)
Got a question?HUD HRE Virtual Help Desk
www.HUDHRE.info
63
New Opportunities
64
Performance Measurement
• Focus of new APR is on reporting on progress madein achieving program level goals
• Not reported until APRs submitted after June 2012,but now is the time to plan for this reporting and workon setting performance measures for your project
• Two types of performance measures in new APR:HUD defined (program type specific), and ProgramDefined
65
Performance Measurement (continued)
• All performance measures reported out on in the APRmust be quantifiable; for Program Defined measures,they do not have to be measured in HMIS (but theymust be able to be systematically measured)
• Grant application process will be the chance to setgoals for the operating year
66
Planning & Implementation: Step 1• Adopt system goals
– Past performance data– Account for state and federal goals (e.g., HPRP, HEARTH)– Account for local goals related to achievement of long- term
impact– Base on uniform, reliable data (e.g. HMIS data elements)
67
Planning and Implementation: Step 2• Establish program performance expectations based on
system goals– Negotiate with agencies– Incorporate into CoC/funding agreements– Include quarterly, semi-annual, annual goals– Response to low performance
68
Planning and Implementation: Step 3• Performance reports
– Set frequency– Determine audience/distribution– Use data that meets quality standards– System and program level performance
69
Tracking System Performance• Programs and community must have an intentional process and
common database to track system outcomes– Use HMIS or integrate HMIS into another community-level database– Integrate other data sources, such as point-in-time street counts or
eviction and housing foreclosure rates, to help measure orcontextualize results
– Ensure data quality
• Consider which data elements are needed to measure theidentified outputs and outcomes
• Consider how you will analyze the data to report outputs andoutcomes and what you will count as “successful”
70
Future Considerations: McKinney Act amended byHEARTH Act focus on Performance Measurement
• Greater focus on CoC-level performance measures – such as…– Reducing the length of time individuals and families remain homeless;– Reduction in the number of times individuals and families experience
homelessness;– Success of outreach to homeless individuals and families;– Overall reduction in the number of homeless individuals and families;– Jobs and income growth for homeless individuals and families;– Success of prevention efforts; and– Success in achieving outcomes related to serving families with children and youth
defined as homeless under other federal statutes.
71
Audience Quiz 6: Why does it matter thatmy program can now report onperformance measures in the APR?
1. It’s what HUD requires grantees to do2. It allows grantees to define, demonstrate, and discuss their
achievements to HUD, as well as other community partners(CoC, TYP, partner agencies, funders)
3. Future HUD funding (HEARTH Act) will push communitiesto show that they can measure the effectiveness of theirwork to prevent and end homelessness
4. Both 2 and 3
72
Summary
• There are many significant changes to the APR thatare designed to allow for more complete, accurate,and meaningful reporting to HUD by its grantees andHPRP grantees
• These changes are being phased in, with theTransition APR being a key first stage
• The use of e-snaps for the APR is aimed at makingreporting easier for grantees, and is a part of HUD’soverall work to automate and go paperless for grantapplication and reporting
73
Summary (continued)
• The new APR provides key opportunities to granteesand communities for how they define and measure theperformance of their services to individuals andfamilies facing homelessness
• Lastly, the HRE website is the source for HUDtraining, technical assistance, list servannouncements, and resources on the APR
74
Questions/Comments?
75
For more information
• Visit the HUD HRE website
– APR page (http://www.HUDHRE.info/apr): Fortraining, guidebooks, and HUD issued guidance
– Virtual Help Desk: To review FAQ’s on the APR,and submit a question to the HUD Help Desk
76
Contact Info
Natalie Matthews, Abt Associates, [email protected]
Louise Rothschild, Abt Associates, [email protected]
Darlene Mathews, Darlene Mathews, [email protected]
