AFP and Telework Program Review U.S. Department of Education OSERS/RSA Rob Groenendaal and Brian...

AFP and Telework AFP and Telework Program ReviewProgram Review

U.S. Department of EducationU.S. Department of Education

OSERS/RSAOSERS/RSA

Rob Groenendaal Rob Groenendaal

and and

Brian Bard Brian Bard

2

What we have learned from the process

• Proof of compliance should not be based on Proof of compliance should not be based on data alone.data alone.

• Every grantee should be assessed the same Every grantee should be assessed the same way but can show compliance in its own way.way but can show compliance in its own way.

• Grantees should not be compared to each Grantees should not be compared to each other. other.

• Compliance should not be black or white.Compliance should not be black or white.• Look for the positive as well as the negative.Look for the positive as well as the negative.

3

The result . . .

• Program Reviews allow for a holistic Program Reviews allow for a holistic explanation of the program (through explanation of the program (through documents, data, and discussion).documents, data, and discussion).• This explanation is done for peer reviewers This explanation is done for peer reviewers and RSA.and RSA.• The peer reviewers and RSA answer a The peer reviewers and RSA answer a series of questions about the program to series of questions about the program to determine the extent of compliance.determine the extent of compliance.

4

The effect . . .

ProsPros• Context is consideredContext is considered• Focuses on what you can controlFocuses on what you can control• Appropriate to size and funding of programAppropriate to size and funding of program

ConsCons• Time-intensive (for all)Time-intensive (for all)• Highly subjectiveHighly subjective• Learning curve for RSA and granteesLearning curve for RSA and grantees

5

Role of Lead Agencies and Role of Lead Agencies and Community Based Organizations in Community Based Organizations in

Program ReviewsProgram Reviews

• The LEAD AGENCY is being reviewed.The LEAD AGENCY is being reviewed.• Participation from BOTH the Lead Agency and Participation from BOTH the Lead Agency and

CBO is expected.CBO is expected.• Participation of all personnel being paid using Participation of all personnel being paid using

grant funds is required (see exception in the next grant funds is required (see exception in the next bullet). bullet).

• Participation of the Certifying Representative is Participation of the Certifying Representative is encouraged but not required.encouraged but not required.

6

How are AFP/Telework Programs selected for review?

Random selection of an equal number of Random selection of an equal number of grantees in two categories:grantees in two categories:(1) Title III AFP/Telework grantees claimed (1) Title III AFP/Telework grantees claimed by a Statewide AT Program as a state financing by a Statewide AT Program as a state financing activity; activity; OROR(2) Title III AFP/Telework grantees (2) Title III AFP/Telework grantees notnot claimed by a Statewide AT Program as a state claimed by a Statewide AT Program as a state financing activity.financing activity.

7

How are AFP/Telework Programs selected for review?

Exceptions to random selection:Exceptions to random selection:• Recent major change in the Recent major change in the AFP/Telework Program or State AFP/Telework Program or State Plan for ATPlan for AT• Recent major disasterRecent major disaster• RSA is concernedRSA is concerned

8

How does a grantee prepare for the review?

A review assesses two “Core A review assesses two “Core Components” of grant implementation, Components” of grant implementation, each in a slightly different way.each in a slightly different way.• Core Component 1 – Program Core Component 1 – Program ManagementManagement• Core Component 2 – Program Core Component 2 – Program PerformancePerformance

9

Core Components of Program Review

Elements of Program Elements of Program ManagementManagement• Fiscal ManagementFiscal Management• Contract OversightContract Oversight• Personnel ManagementPersonnel Management

Elements of Program Elements of Program PerformancePerformance• Emphasis and Expansion Emphasis and Expansion of Consumer Choice and of Consumer Choice and ControlControl• Timely and Equitable Timely and Equitable Access to Financial Access to Financial AssistanceAssistance• Capacity to Continue on a Capacity to Continue on a Permanent BasisPermanent Basis

10

Program Management

The review of Program Management verifies The review of Program Management verifies that a grant is managed efficiently and that a grant is managed efficiently and appropriately in accordance with title III, the appropriately in accordance with title III, the NFP, EDGAR, and OMB Circulars, as NFP, EDGAR, and OMB Circulars, as applicable. This includes:applicable. This includes:• Fiscal Management Fiscal Management • Contract OversightContract Oversight• Personnel ManagementPersonnel Management

11

Fiscal ManagementEDGAR requirementsEDGAR requirements::• Fiscal control and accounting procedures to insure proper Fiscal control and accounting procedures to insure proper disbursement of and accounting for Federal funds; controls and disbursement of and accounting for Federal funds; controls and procedures adequate to establish that such funds have not been procedures adequate to establish that such funds have not been used in violation of applicable statutes. used in violation of applicable statutes. • Records that fully show the amount of funds under the grant, Records that fully show the amount of funds under the grant, how the grantee uses the funds, the total cost of the project, the how the grantee uses the funds, the total cost of the project, the share of the cost provided by other sources, and other records to share of the cost provided by other sources, and other records to facilitate an effective audit.facilitate an effective audit.• Audits in accordance with OMB A-133.Audits in accordance with OMB A-133.

12

Fiscal ManagementTitle III or NFP requirementsTitle III or NFP requirements::• Assurance that the grantee will provide the non-federal share Assurance that the grantee will provide the non-federal share of the cost of the program in cash, from state, local, or private of the cost of the program in cash, from state, local, or private sources.sources.• Assurance that the grant funds will be used to supplement and Assurance that the grant funds will be used to supplement and not supplant other federal, state, and local public funds.not supplant other federal, state, and local public funds.• Assurance that the grant funds will be placed in a permanent Assurance that the grant funds will be placed in a permanent separate account and identified and accounted for separately separate account and identified and accounted for separately from any other fund.from any other fund.• Assurance that the funds are administered, expended, and Assurance that the funds are administered, expended, and invested in accordance with title III or the NFP.invested in accordance with title III or the NFP.

13

How to prepare for a review of Fiscal Management

Submit to RSASubmit to RSA::• Expenditure report.Expenditure report.• Proof of title III or NFP fiscal management assurances.Proof of title III or NFP fiscal management assurances.• Approved indirect cost rate agreement/cost allocation plan or Approved indirect cost rate agreement/cost allocation plan or questionnaire provided by RSA (applies to both Lead Agencies and questionnaire provided by RSA (applies to both Lead Agencies and Community Based Organizations.)Community Based Organizations.)• Copies of most recent audit reports or questionnaire provided by Copies of most recent audit reports or questionnaire provided by RSA (applies to both Lead Agencies and CBOs).RSA (applies to both Lead Agencies and CBOs).

(In addition, RSA will look at ED 524Bs and GAPS records.)(In addition, RSA will look at ED 524Bs and GAPS records.)

14

Expenditure ReportExpenditure Report

• 1 for Lead Agency to administer the grant, 1 for CBO to 1 for Lead Agency to administer the grant, 1 for CBO to implement the program.implement the program.• From the most recent federal fiscal year.From the most recent federal fiscal year.• There is no “format”, but RSA must be able to understand the There is no “format”, but RSA must be able to understand the reports.reports.• Show use of all federal and match funds as well as income from Show use of all federal and match funds as well as income from interest, investments, loan repayments, and other sources.interest, investments, loan repayments, and other sources.• Show use of funds used for administration of the program Show use of funds used for administration of the program separately from the amount used for loans and other program separately from the amount used for loans and other program purposes.purposes.• Show the amount of funds used for indirect costs.Show the amount of funds used for indirect costs.

15

Expenditure ReportExpenditure Report

What RSA is looking forWhat RSA is looking for::• Existence, quality, and legitimacy Existence, quality, and legitimacy of reportof report• Permanent separate accountPermanent separate account• Compliance with indirect cost Compliance with indirect cost limitslimits

16

Proof of Fiscal Management AssurancesProof of Fiscal Management Assurances

What RSA is looking forWhat RSA is looking for::• The sources, amounts, and appropriateness of match The sources, amounts, and appropriateness of match provided for all grants for the AFP/Telework Program. provided for all grants for the AFP/Telework Program. • The Lead Agency’s and CBO’s internal controls that The Lead Agency’s and CBO’s internal controls that ensure all principal and interest from the permanent ensure all principal and interest from the permanent separate account (a) support only AFP/Telework separate account (a) support only AFP/Telework activities and (b) supplement and do not supplant other activities and (b) supplement and do not supplant other federal, state, and local public funds expended to federal, state, and local public funds expended to provide financial loans.provide financial loans.

17

Indirect Cost Rates

What RSA is looking forWhat RSA is looking for::• Existence of agreementExistence of agreement• Appropriate kind of agreementAppropriate kind of agreement• Handling of indirect costs between Lead Handling of indirect costs between Lead Agency and CBOAgency and CBO• Amount of indirect will be seen in Amount of indirect will be seen in expenditure reportsexpenditure reports

18

Audits

What RSA is looking forWhat RSA is looking for::

• An audit was done (if required)An audit was done (if required)• Findings of the auditFindings of the audit• Resolution of findingsResolution of findings

19

ED 524B Forms and G5/GAPS Records

What RSA is looking forWhat RSA is looking for::• Completeness and accuracy of Completeness and accuracy of

financial reportsfinancial reports

20

Contract Oversight

EDGAR requires grantees toEDGAR requires grantees to::• Directly administer or supervise the administration Directly administer or supervise the administration

of each project.of each project.• Have procedures for providing TA, evaluation and Have procedures for providing TA, evaluation and

for performing other administrative for performing other administrative responsibilities necessary to ensure compliance.responsibilities necessary to ensure compliance.

• Monitor activities to assure compliance with Monitor activities to assure compliance with applicable Federal requirements and that applicable Federal requirements and that performance goals are being achieved.performance goals are being achieved.

21

Contract Oversight

Grantees are required to enter into a contract with a CBO or Grantees are required to enter into a contract with a CBO or group of CBOs that has individuals with disabilities group of CBOs that has individuals with disabilities involved in the organizational decision-making to involved in the organizational decision-making to administer the AFP/Telework Program. The contract must:administer the AFP/Telework Program. The contract must:

• Include a provision requiring funds be administered Include a provision requiring funds be administered consistent with title III or the NFP.consistent with title III or the NFP.

• Include any provision the Secretary requires concerning Include any provision the Secretary requires concerning oversight and evaluation of federal financial interests.oversight and evaluation of federal financial interests.

• Require the CBO(s) enter into a contract with commercial Require the CBO(s) enter into a contract with commercial lending institutions or organizations or state financing lending institutions or organizations or state financing agencies.agencies.

22

How to prepare for a review of Contract Oversight

Submit to RSASubmit to RSA: : • A dated copy of the grantee’s most recent A dated copy of the grantee’s most recent contract with the CBO contract with the CBO

23

How to prepare for a review of Contract Oversight

What RSA is looking forWhat RSA is looking for::• Appropriate oversight to protect Federal Appropriate oversight to protect Federal interests and ensure implementationinterests and ensure implementation• Relevant to title III or the NFPRelevant to title III or the NFP

24

Personnel Management

OMB CircularsOMB Circulars::• Compensation to personnel must be reasonable.Compensation to personnel must be reasonable.• Charges to Federal awards for salaries and wages are Charges to Federal awards for salaries and wages are

based on documented payrolls. based on documented payrolls. • Charges for salaries and wages of employees who work Charges for salaries and wages of employees who work

solely on a single Federal award are supported by periodic solely on a single Federal award are supported by periodic certifications.certifications.

• Charges for salaries and wages of employees who work on Charges for salaries and wages of employees who work on multiple activities are supported by personnel activity multiple activities are supported by personnel activity reports.reports.

25

How to prepare for a review of Personnel Management

Submit to RSA eitherSubmit to RSA either::

(A)(A) Position descriptions of record for those Position descriptions of record for those being paid with grant funds and personnel being paid with grant funds and personnel activity reports for them; ORactivity reports for them; OR

(B)(B) Synopsis for key personnel Synopsis for key personnel describing his describing his or her role and responsibilities or her role and responsibilities associated associated with the grantwith the grant..

26

Personnel Management

What RSA is looking forWhat RSA is looking for::• Appropriate number of personnelAppropriate number of personnel• Appropriate time commitments of Appropriate time commitments of

personnelpersonnel• Appropriate duties for personnelAppropriate duties for personnel

27

How to prepare for a review of Program Management

• Submit all the previously described documents.Submit all the previously described documents.• RSA reviews the documents.RSA reviews the documents.• RSA discusses the documents via conference call RSA discusses the documents via conference call

(the grantee is responsible for bringing together (the grantee is responsible for bringing together the appropriate parties).the appropriate parties).

• RSA uses this information to answer questions RSA uses this information to answer questions about program management.about program management.

• Yes/no with explanation.Yes/no with explanation.

28

Questions about Program Management

1. Has the grantee obtained its full matching funds from 1. Has the grantee obtained its full matching funds from appropriate sources? If not, does the grantee have a plan for appropriate sources? If not, does the grantee have a plan for obtaining the remaining match funds? obtaining the remaining match funds?

2. Does the grantee account for grant funds accurately and 2. Does the grantee account for grant funds accurately and completely in accordance with EDGAR? completely in accordance with EDGAR?

3. Does the grantee use grant funds only for purposes under, 3. Does the grantee use grant funds only for purposes under, and within the limitations of, title III or the NFP, as and within the limitations of, title III or the NFP, as applicable?applicable?

4. Does the grantee comply with indirect cost requirements?4. Does the grantee comply with indirect cost requirements?

5. Does the grantee comply with audit requirements?5. Does the grantee comply with audit requirements?

29


6. Does the grantee exercise appropriate fiscal and 6. Does the grantee exercise appropriate fiscal and performance oversight of the AFP/Telework Program through performance oversight of the AFP/Telework Program through its contract with the CBO? If not, does the Lead Agency its contract with the CBO? If not, does the Lead Agency have a plan for improving its oversight?have a plan for improving its oversight?

7. Is the contract structured to meet the requirements and 7. Is the contract structured to meet the requirements and intent of title III of the NFP, as applicable? If note does the intent of title III of the NFP, as applicable? If note does the Lead Agency have a plan for changing its contract to make it Lead Agency have a plan for changing its contract to make it more consistent with title III or the NFP, as applicable?more consistent with title III or the NFP, as applicable?

30


8. Does the AFP/Telework Program have an appropriate number of 8. Does the AFP/Telework Program have an appropriate number of staff to implement the grant? If not, does the Lead Agency have a staff to implement the grant? If not, does the Lead Agency have a plan for changing its contract to make it more consistent with title III plan for changing its contract to make it more consistent with title III or the NFP, as applicable?or the NFP, as applicable?

9. Do the Lead Agency and CBO distribute the time of their 9. Do the Lead Agency and CBO distribute the time of their personnel appropriately to implement the grant based on their personnel appropriately to implement the grant based on their responsibilities? If not, does the grantee have a plan to distribute time responsibilities? If not, does the grantee have a plan to distribute time more appropriately?more appropriately?

10. Do the Lead Agency and CBO use grant funds to support only 10. Do the Lead Agency and CBO use grant funds to support only personnel with responsibilities germane to the grant?personnel with responsibilities germane to the grant?

31

Assessing Compliance of Program Management

There is no overall rating of compliance.There is no overall rating of compliance.

For each element of Program Management, a For each element of Program Management, a grantee will receive 1 of 3 ratings:grantee will receive 1 of 3 ratings:

• CompliantCompliant• In Need of ImprovementIn Need of Improvement• Fails to Substantially ComplyFails to Substantially Comply

32


The rating is based on how RSA answers the The rating is based on how RSA answers the questions:questions:

• Initial question: Does the AFP/Telework Initial question: Does the AFP/Telework Program have an appropriate number of staff Program have an appropriate number of staff to implement the grant? to implement the grant? • Subquestion: If not, does the grantee have a Subquestion: If not, does the grantee have a plan for adjusting the number of staff? plan for adjusting the number of staff?

33


• If the answer to the initial question is “Yes” – the If the answer to the initial question is “Yes” – the grantee is rated as compliant.grantee is rated as compliant.

• If the answer to the initial question is “No” – the If the answer to the initial question is “No” – the rating depends on the answer to the subquestion.rating depends on the answer to the subquestion.

• If the answer to the subquestion is “Yes” – the If the answer to the subquestion is “Yes” – the grantee is rated as in need of improvement.grantee is rated as in need of improvement.

• If the answer to the subquestion is “No” – the If the answer to the subquestion is “No” – the grantee is rated as failing to substantially comply.grantee is rated as failing to substantially comply.

34


Questions for which “needs improvement” doesn’t apply:Questions for which “needs improvement” doesn’t apply:

• Does the grantee account for grant funds accurately and Does the grantee account for grant funds accurately and completely in accordance with EDGAR? completely in accordance with EDGAR?

• Does the grantee use grant funds only for purposes under, and Does the grantee use grant funds only for purposes under, and within the limitations of, title III or the NFP, as applicable?within the limitations of, title III or the NFP, as applicable?

• Does the grantee comply with indirect cost requirements?Does the grantee comply with indirect cost requirements?• Does the grantee comply with audit requirements?Does the grantee comply with audit requirements?• Does the grantee use grant funds to support only personnel with Does the grantee use grant funds to support only personnel with

responsibilities germane to the grant? responsibilities germane to the grant?

35


The onus is on the grantee to present The onus is on the grantee to present evidence/plan sufficient to get to a “yes” answer.evidence/plan sufficient to get to a “yes” answer.

• RSA will ask questions or provide examples RSA will ask questions or provide examples of sufficient evidence, if needed.of sufficient evidence, if needed.• Grantee likely will know their status before Grantee likely will know their status before the conversation is over.the conversation is over.

36

Program Performance

The review of Program Performance assesses whether The review of Program Performance assesses whether the AFP/Telework Program is implementing its grant in the AFP/Telework Program is implementing its grant in accordance with title III or the NFP, as applicable. This accordance with title III or the NFP, as applicable. This includes:includes:• Extent to which the program emphasizes and expands Extent to which the program emphasizes and expands consumer choice and control.consumer choice and control.• Extent to which program provides timely and equitable Extent to which program provides timely and equitable access to financial assistance.access to financial assistance.• Extent to which the program can continue on a Extent to which the program can continue on a permanent basis. permanent basis.

37

Third-Party Reviewers

Peer reviewers participate in the review of Program PerformancePeer reviewers participate in the review of Program Performance::• 3 current or former program directors (or position with 3 current or former program directors (or position with similar duties).similar duties).• At least 1 with prior review experience on each team.At least 1 with prior review experience on each team.• Trained, paid volunteers from a standing pool.Trained, paid volunteers from a standing pool.• No reviewers from RESNA AT Catalyst or NISAT.No reviewers from RESNA AT Catalyst or NISAT.• Participation is not anonymous, but conclusions are not Participation is not anonymous, but conclusions are not attributed to individual reviewers.attributed to individual reviewers.

Peer reviewers learn about the program through review of documents, Peer reviewers learn about the program through review of documents, looking at data, and discussion on teleconferences/webinars.looking at data, and discussion on teleconferences/webinars.

38

How to prepare for a review of Program Performance

Submit to RSASubmit to RSA::• Narrative explaining how consumer choice and control is Narrative explaining how consumer choice and control is emphasized and expanded.emphasized and expanded.• Copies of administrative policies and procedures manuals. Copies of administrative policies and procedures manuals. • Copies of lender agreement(s), investment plan, if Copies of lender agreement(s), investment plan, if applicable, and narrative or plan for sustainability. applicable, and narrative or plan for sustainability. • Reviewers already will have AFP/Telework Program data.Reviewers already will have AFP/Telework Program data.

There is no prescribed format for these but must be There is no prescribed format for these but must be accessible.accessible.

39


• Develop teleconference/webinar Develop teleconference/webinar presentations that serve as a virtual on-site presentations that serve as a virtual on-site visit to your program.visit to your program.• Length, format, and content of presentation Length, format, and content of presentation are up to the program.are up to the program.• The “presenter” during a teleconference or The “presenter” during a teleconference or webinar is up to the program. webinar is up to the program. • Need to be accessible.Need to be accessible.

40

Emphasis and Expansion of Consumer Choice and Control

Title III or NFP requirementsTitle III or NFP requirements::• Assurance that, and information describing Assurance that, and information describing the manner in which, the grantee will expand the manner in which, the grantee will expand and emphasize consumer choice and control. and emphasize consumer choice and control.

41

How to prepare for a review of Consumer Choice and Control

Submit to RSASubmit to RSA::• A narrative describing how individuals with A narrative describing how individuals with disabilities are involved in organizational disabilities are involved in organizational decision-making at all organizational levels to decision-making at all organizational levels to administer the loan program.administer the loan program.

42

How to prepare for a review of Consumer Choice and Control

What RSA is looking forWhat RSA is looking for::• Appropriate emphasis and expansion of Appropriate emphasis and expansion of involvement in organizational decision-involvement in organizational decision-making by individuals with disabilities making by individuals with disabilities • The grantee should also be prepared to The grantee should also be prepared to discuss consumer choice and control during discuss consumer choice and control during webinars/teleconferenceswebinars/teleconferences

43

Timely and Equitable Access to Financial Assistance

Title III or NFP requirementsTitle III or NFP requirements::• Assurance that the grantee enter into a contract Assurance that the grantee enter into a contract

with a CBO that has policies and procedures to:with a CBO that has policies and procedures to:(1)(1) Review and process requests for financial Review and process requests for financial

assistance in a timely manner.assistance in a timely manner.(2)(2) Ensure equitable access to all consumers for which Ensure equitable access to all consumers for which

financing is requested through the program.financing is requested through the program.(3)(3) Ensure consumer-controlled oversight of the Ensure consumer-controlled oversight of the

program. program.

44

How to prepare for a review of Timely and Equitable Access to Financial Assistance

Submit to RSASubmit to RSA::• A dated copy of its most recent revised and A dated copy of its most recent revised and adopted AFP/Telework policies and procedures adopted AFP/Telework policies and procedures manual.manual.• A dated copy of its most recent revised and A dated copy of its most recent revised and adopted lender agreement(s).adopted lender agreement(s).

45

How to prepare for a review of Timely and Equitable Access to Financial Assistance

What RSA is looking forWhat RSA is looking for::• Appropriate access to financial assistance.Appropriate access to financial assistance.• The grantee should also be prepared to The grantee should also be prepared to discuss how the program assures the review discuss how the program assures the review and process of financial loans in a timely and process of financial loans in a timely manner and describe the flow of an manner and describe the flow of an application for an AFP/Telework loan during application for an AFP/Telework loan during teleconferences or webinars. teleconferences or webinars.

46

Capacity to Continue on a Permanent Basis

Title III or NFP requirementsTitle III or NFP requirements::• Assurance that the AFP/Telework Program will:Assurance that the AFP/Telework Program will:

(1)(1) Continue on a permanent basis.Continue on a permanent basis.

(2)(2) Invest the funds within the permanent separate Invest the funds within the permanent separate account in low-risk securities.account in low-risk securities.

(3)(3) Administer the funds with prudence, discretion, Administer the funds with prudence, discretion, and intelligence.and intelligence.

47

How to prepare for a review of Capacity to Continue on a Permanent Basis

Submit to RSASubmit to RSA::• A dated copy of its investment plan, if applicable, or A dated copy of its investment plan, if applicable, or narrative describing how loan program funds are narrative describing how loan program funds are invested.invested.• A copy of its investment portfolio.A copy of its investment portfolio.• A dated copy of the its plan for sustainability, if one A dated copy of the its plan for sustainability, if one has been developed, or narrative explaining how the has been developed, or narrative explaining how the program will continue on a permanent basis.program will continue on a permanent basis.

48

Investment Plan

What RSA is looking forWhat RSA is looking for::• Appropriate investment of AFP/Telework Appropriate investment of AFP/Telework Program funds in the permanent separate Program funds in the permanent separate account.account.• The grantee should also be prepared to discuss The grantee should also be prepared to discuss the implementation and regular revision of the the implementation and regular revision of the AFP/Telework Program’s investment plan.AFP/Telework Program’s investment plan.

49

Sustainability Plan

What RSA is looking forWhat RSA is looking for::• Appropriate implementation of strategies Appropriate implementation of strategies for sustaining the AFP/Telework Program.for sustaining the AFP/Telework Program.• The grantee should also be prepared to The grantee should also be prepared to discuss how the program is implementing a discuss how the program is implementing a plan for sustainability during a plan for sustainability during a webinar/teleconference.webinar/teleconference.

50


Based on documents, data, and discussion, third-Based on documents, data, and discussion, third-party reviewers respond to a series of questions: party reviewers respond to a series of questions: • Yes/no with explanation.Yes/no with explanation.• Evidence can come from any source of the review.Evidence can come from any source of the review.• Getting a “yes” answer does not depend on an Getting a “yes” answer does not depend on an AFP/Telework Program having any particular AFP/Telework Program having any particular structure or type. structure or type. • A “no” answer does not automatically mean a A “no” answer does not automatically mean a grantee is out of compliance.grantee is out of compliance.

51

Questions about Program Performance

1. Is there evidence that individuals with disabilities are involved in 1. Is there evidence that individuals with disabilities are involved in the administrative decision-making of the AFP/Telework Program?the administrative decision-making of the AFP/Telework Program?

a) If not, does the grantee provide justification for the current level a) If not, does the grantee provide justification for the current level of involvement of individuals with disabilities in the administrative of involvement of individuals with disabilities in the administrative decision-making of the loan program?decision-making of the loan program?

b) If not, does the grantee have a reasonable plan for improving the b) If not, does the grantee have a reasonable plan for improving the involvement of individuals with disabilities in the administrative involvement of individuals with disabilities in the administrative decision-making of the loan program?decision-making of the loan program?

52


2. Is there evidence that the AFP/Telework Program 2. Is there evidence that the AFP/Telework Program emphasizes and expands consumer choice and control? emphasizes and expands consumer choice and control?

a) If not, does the grantee provide justification for why the a) If not, does the grantee provide justification for why the current level of consumer choice and control is appropriate?current level of consumer choice and control is appropriate?

b) If not, does the grantee have a reasonable plan for improving b) If not, does the grantee have a reasonable plan for improving its emphasis and expansion of consumer choice and control?its emphasis and expansion of consumer choice and control?

53


3. Is there evidence that the policies and procedures of the 3. Is there evidence that the policies and procedures of the AFP/Telework Program make it accessible to consumers regardless of AFP/Telework Program make it accessible to consumers regardless of type of disability, age, income level, type of AT device, equipment, or type of disability, age, income level, type of AT device, equipment, or service for which financing is requested through the program? service for which financing is requested through the program?

a) If not, does the grantee provide justification for why the program is not a) If not, does the grantee provide justification for why the program is not accessible to all individuals with disabilities in the State?accessible to all individuals with disabilities in the State?

b) If not, does the grantee have a reasonable plan for improving the b) If not, does the grantee have a reasonable plan for improving the accessibility of the program?accessibility of the program?

54


4. Is there evidence that the structure and practices of the 4. Is there evidence that the structure and practices of the AFP/Telework Program make it possible to meet the needs of AFP/Telework Program make it possible to meet the needs of individuals in most areas of the State? individuals in most areas of the State?

a) If not, does the grantee provide justification for why the areas a) If not, does the grantee provide justification for why the areas reached in the State are sufficient?reached in the State are sufficient?

b) If not, does the grantee have a reasonable plan for improving b) If not, does the grantee have a reasonable plan for improving their ability to reach consumers in other areas of the State?their ability to reach consumers in other areas of the State?

55


5. Is the lender agreement structured to promote the goals 5. Is the lender agreement structured to promote the goals of the AFP/Telework Program? of the AFP/Telework Program?

a) If not, does the grantee provide justification for the a) If not, does the grantee provide justification for the structure of the current lender agreement?structure of the current lender agreement?

b) If not, does the grantee have a reasonable plan for b) If not, does the grantee have a reasonable plan for improving the lender agreement? improving the lender agreement?

56


6. Does the AFP/Telework Program administer its funds 6. Does the AFP/Telework Program administer its funds with prudence, discretion, and intelligence? with prudence, discretion, and intelligence?

a) If not, does the grantee provide justification for why the a) If not, does the grantee provide justification for why the program administers its funds in the current manner?program administers its funds in the current manner?

b) If not, does the grantee have a reasonable plan for b) If not, does the grantee have a reasonable plan for improving the administration of program funds? improving the administration of program funds?

57


7. Does the AFP/Telework Program invest funds in low-risk 7. Does the AFP/Telework Program invest funds in low-risk securities in which a regulated insurance company may invest securities in which a regulated insurance company may invest under the law of the State? under the law of the State?

a) If not, does the grantee provide justification for why the a) If not, does the grantee provide justification for why the program invests in the manner it currently does?program invests in the manner it currently does?

b) If not, does the grantee have a reasonable plan for investing b) If not, does the grantee have a reasonable plan for investing program funds more appropriately? program funds more appropriately?

58


8. Does the AFP/Telework Program have an investment plan that 8. Does the AFP/Telework Program have an investment plan that is implemented and revised regularly?is implemented and revised regularly?

a) If not, does the grantee provide justification for why the a) If not, does the grantee provide justification for why the program has not implemented an investment plan that is reviewed program has not implemented an investment plan that is reviewed regularly?regularly?

b) If not, does the grantee have a reasonable plan for the b) If not, does the grantee have a reasonable plan for the implementation and regular revision of an investment plan? implementation and regular revision of an investment plan?

59


9. Is the AFP/Telework Program implementing 9. Is the AFP/Telework Program implementing appropriate strategies for sustainability?appropriate strategies for sustainability?

a) If not, does the grantee provide justification for a) If not, does the grantee provide justification for why the program is not implementing appropriate why the program is not implementing appropriate strategies for sustainability?strategies for sustainability?

b) If not, does the grantee have a plan for b) If not, does the grantee have a plan for sustainability that could be implemented? sustainability that could be implemented?

60

Assessing Compliance of Program Performance

There is no overall rating of compliance.There is no overall rating of compliance.

For each element of Program Performance, a For each element of Program Performance, a grantee will receive 1 of 3 ratings:grantee will receive 1 of 3 ratings:

• CompliantCompliant• In Need of ImprovementIn Need of Improvement• Fails to Substantially ComplyFails to Substantially Comply

61


The rating is based on how peer reviewers answer the questionsThe rating is based on how peer reviewers answer the questions::• Initial question: Is there evidence that the structure and Initial question: Is there evidence that the structure and practices of the AFP/Telework Program makes it possible to practices of the AFP/Telework Program makes it possible to meet the needs of individuals with disabilities in most areas meet the needs of individuals with disabilities in most areas of the State? of the State? • Subquestion 1: If not, does the grantee provide justification Subquestion 1: If not, does the grantee provide justification for why the areas reached by the program in the State for why the areas reached by the program in the State currently are sufficient? currently are sufficient? • Subquestion 2: If not, does the grantee have a reasonable Subquestion 2: If not, does the grantee have a reasonable plan for improving the capacity of the program to reach plan for improving the capacity of the program to reach individuals in more areas of the State? individuals in more areas of the State?

62


• If the answer to the initial question is “Yes” – the grantee is rated as If the answer to the initial question is “Yes” – the grantee is rated as compliant.compliant.

• If the answer to the initial question is “No” – the rating depends on the If the answer to the initial question is “No” – the rating depends on the answer to the subquestions.answer to the subquestions.

• If the answer to subquestion 1 is “Yes” – the grantee is rated as If the answer to subquestion 1 is “Yes” – the grantee is rated as compliant.compliant.

• If the answer to subquestion 1 is “No” – the the rating depends on the If the answer to subquestion 1 is “No” – the the rating depends on the answer to subquestion 2.answer to subquestion 2.

• If the answer to subquestion 2 is “Yes” – the grantee is rated as in If the answer to subquestion 2 is “Yes” – the grantee is rated as in need of improvement.need of improvement.

• If the answer to subquestion 2 is “No” – the grantee is rated as failing If the answer to subquestion 2 is “No” – the grantee is rated as failing to substantially comply.to substantially comply.

63


The onus is on the grantee to present The onus is on the grantee to present evidence/plan sufficient to get to a “yes” answer.evidence/plan sufficient to get to a “yes” answer.

• RSA and third party reviewers will ask RSA and third party reviewers will ask questions or provide examples of sufficient questions or provide examples of sufficient evidence if needed.evidence if needed.• Grantee likely will know their status before Grantee likely will know their status before the conversation is over.the conversation is over.

64

Program DataProgram Data

• AFP/Telework Program data will not determine AFP/Telework Program data will not determine compliance.compliance.• Data will be discussed during a teleconference Data will be discussed during a teleconference or webinar and reflected in the report: or webinar and reflected in the report:

(1)(1) Grantees strengths and areas in need of Grantees strengths and areas in need of improvement when it comes to data; and improvement when it comes to data; and

(2)(2) How the grantee is learning from and using How the grantee is learning from and using its data.its data.

65


Data collection infrastructure:Data collection infrastructure:• How the data is collected.How the data is collected.• How the data is checked.How the data is checked.• Strengths and areas in need of Strengths and areas in need of improvement.improvement.

66


Understanding your data:Understanding your data:• How the nature of your state and structure How the nature of your state and structure

of your program influence the data.of your program influence the data.• What your data tells you about the program What your data tells you about the program

and how it influences implementation.and how it influences implementation.• Strengths and areas in need of Strengths and areas in need of

improvement. improvement.

67


Activities of peer reviewers:Activities of peer reviewers:• Read documents.Read documents.• Participate in teleconferences/webinars.Participate in teleconferences/webinars.• Complete review forms (answer questions).Complete review forms (answer questions).• Meet as a group with RSA to discuss Meet as a group with RSA to discuss

results.results.• Finalize review forms and submit to RSA.Finalize review forms and submit to RSA.

68

Concluding the Program Review

• RSA compiles Program Management and Program RSA compiles Program Management and Program Performance information into a comprehensive report.Performance information into a comprehensive report.• Review of both AFP and Telework grants, if Review of both AFP and Telework grants, if applicable, are consolidated into one report summarizing applicable, are consolidated into one report summarizing findings, recommendations, and highlighting differences. findings, recommendations, and highlighting differences. • RSA shares report with grantee (on-site available).RSA shares report with grantee (on-site available).• RSA finalizes and receives approval of report.RSA finalizes and receives approval of report.• Grantee may respond to report in writing.Grantee may respond to report in writing.

69

Concluding the Program Review: Report

IntroductionIntroduction

Executive SummaryExecutive Summary

Review of Program Review of Program ManagementManagement

• Fiscal Management Fiscal Management • Contract Oversight Contract Oversight • Personnel Personnel

Management Management

Review of Program PerformanceReview of Program Performance• Emphasis and Expansion of Emphasis and Expansion of

Consumer Choice and Consumer Choice and ControlControl

• Timely and Equitable Timely and Equitable Access to Financial Access to Financial Assistance Assistance

• Capacity to Continue on a Capacity to Continue on a Permanent BasisPermanent Basis

DataData

ConclusionConclusion

AppendicesAppendices

70

Outcomes of a Review

Compliant = No action necessary.Compliant = No action necessary.In Need of Improvement = AFP/Telework grantee In Need of Improvement = AFP/Telework grantee implements its own plan for improvement.implements its own plan for improvement.Failure to Substantially Comply = Failure to Substantially Comply = • Prior to imposing sanctions, RSA and grantee will Prior to imposing sanctions, RSA and grantee will negotiate corrective action plan. negotiate corrective action plan. • Possible referral for TA from NATTAP or NISAT.Possible referral for TA from NATTAP or NISAT.• Failure to implement corrective action may result Failure to implement corrective action may result in sanctions. in sanctions.

Date post:	29-Dec-2015
Category:	Documents
Upload:	briana-shanna-butler
View:	214 times
Download:	0 times

AFP and Telework Program Review U.S. Department of Education OSERS/RSA Rob Groenendaal and Brian...

Documents