Post on 31-May-2018
transcript
8/14/2019 description: tags: Session12
1/37
8/14/2019 description: tags: Session12
2/37
Session 12 2
Session 12
Which? Who? How? and SoWhat?:
Using the Quality AnalysisTool to Answer Your
Institutional VerificationQuestions
8/14/2019 description: tags: Session12
3/37
Session 12 3
How can I use the
Quality Analysis Tool?
Which FAFSA fields change the mostafter initial application? Who is the mostlikely to make changes? How caninstitutional verification profiles beassessed? The session describes how toanswer these questions with the Quality
Analysis Tool and improve yourverification procedures.
8/14/2019 description: tags: Session12
4/37
Session 12 4
Roadmap of Online
Resourcesn SFA download
(http://www.sfadownload.ed.gov/index.htm)
The software Install guide and desk reference
n QA Program website (http://qaprogram.air.org)
Query Guidance
Web cast videosActivity Guides
8/14/2019 description: tags: Session12
5/37
Session 12 5
The Question Catch 22
n Parents and teachers know
n Hamlet tragically found out
n The dilemmaAction
Asking
n
When dealing with othersWhy didnt you ask me?Why dont you just tell me what to do?
8/14/2019 description: tags: Session12
6/37
Session 12 6
Question Quotes
n It is better to know some of the questions thanall of the answers. James Thurber
n
The important thing is not to stop questioning.Albert Einstein
n In all affairs it's a healthy thing now and then tohang a question mark on the things you havelong taken for granted. Bertrand Russell
n A young man is embarrassed to question anolder one. Homer
8/14/2019 description: tags: Session12
7/37
Session 12 7
The Quality Analysis Tool(a.k.a. ISIR Analysis Tool)
n Formulate your own questions
n Find answers in your ISIR data
n Figure out how to improve your institutionalverification procedures
8/14/2019 description: tags: Session12
8/37
Session 12 8
Institutional
Verificationn Every institution has their own institutional
verification system
n
Many non-QA Program schools verify applicantsin addition to those selected by the centralprocessing system (CPS)
n Only institutions formally participating in the QAProgram are exempt from following the
prescribed CPS editsn Every school can analyze what they are doing
8/14/2019 description: tags: Session12
9/37
Session 12 9
How do you start?
n Do you want to improve your institutionalverification process?
n Retain or locate initial and paid on ISIRinformation
SARA files
EDExpressMainframe systems that can write SARA
records
8/14/2019 description: tags: Session12
10/37
Session 1210
Prerequisite Steps
n Download and install software
n Import initial and paid on data
n Write queries that mirror your institutionsverification profiles - Query Guidanceonline
n Remember where your data came from
8/14/2019 description: tags: Session12
11/37
Session 1211
Queries
n Select cases if a given logical condition ismet
n Same query engine as used in EDExpress
n QA Tool Demonstration
CPS Selected for Verification
Parents AGI less than twice the sum ofparents worksheets A and B
8/14/2019 description: tags: Session12
12/37
Session 1212
EstablishingInstitutionalVerification Profiles
n Tools | Setup | Verification Edit Profile
n Process | Verification Selection | Set InitialSchool Verification Flag
8/14/2019 description: tags: Session12
13/37
Session 1213
Analysis Options
n Which ISIR fields change?
n Who makes changes?
n How do you assess verification profiles?
n And more!
8/14/2019 description: tags: Session12
14/37
Session 1214
Analysis Guidance on
qaprogram.air.orgn Activity Guide #1 ISIR field change
n Activity Guide #2 which applicants
n Activity Guide #3 assessing profiles
n Web cast overview and introduction
8/14/2019 description: tags: Session12
15/37
Session 1215
Which ISIR Fields
Change?n Identify the most problematic ISIR data
elements your verification process needs to
addressn Help determine which information is most
important to confirm with applicantsselected for verification
n Identify areas that may benefit fromimproved education efforts
8/14/2019 description: tags: Session12
16/37
Session 1216
Steps for ISIR Field
Change Analysisn Plan Analysis
Do you want to limit the number of ISIR fieldsexamined?
Do you want to examine impact upon EFC or PellGrant eligibility?
Do you want to analyze dependent and independentstudents separately?
n
Separate verified and non-verified casesBuild new query using School Verification FlagUse as (or with other) selection criteria in all reports
8/14/2019 description: tags: Session12
17/37
Session 1217
EFC-Focus: Field ChangeReport with CorrectedField Counts
n Counts of records with a changes to specific ISIRFields
n Counts of records with the specific changeexperiencing an EFC increase
n Counts of records with the specific changeexperiencing an EFC decrease
n Counts of records with the specific change notexperiencing an EFC change
n Unduplicated corrected and non-corrected recordcounts
8/14/2019 description: tags: Session12
18/37
8/14/2019 description: tags: Session12
19/37
Session 1219
Pell Focus: CorrectedRecords within an InitialEFC Range Report
n Similar to Field Change Report, but focus is on thedeparture from specified EFC ranges
n Contextual meaning of EFC decrease andincrease change from initial range
n Use Initial Pell eligible as the initial EFC range toexamine which ISIR elements are most associatedwith the loss of Pell Grant eligibility
n Use Initial Pell ineligible as the initial EFC range toexamine which ISIR elements are most associatedwith gaining Pell Grant eligibility
8/14/2019 description: tags: Session12
20/37
Session 1220
What to Look for in theCorrected Records Report
n Initial Pell eligible versionThe ISIR fields most associated with EFC
increase (loss of eligibility)Over-award summary statistics
n Initial Pell ineligible versionThe ISIR fields most associated with EFC
decreases (gain of eligibility)
Under-award summary statistics
8/14/2019 description: tags: Session12
21/37
Session 1221
Questions to follow
either report
n How does your institution currently address the
ISIR elements identified as being the most
problematic?
n Are you missing the same types of errors in your
non-verified population?
n Could education efforts reduce these errors?
n Is there a way to reduce the number of non-
corrected records selected for verification?*
8/14/2019 description: tags: Session12
22/37
Session 1222
Who Makes Changes?
n Identify ranges where changes to ISIRelements are most pronounced
n Identify ranges where changes to ISIRinformation are most associated with EFCincreases
n Identify ranges where changes to ISIR
information are most associated with EFCdecreases
8/14/2019 description: tags: Session12
23/37
Session 1223
Steps to take beforeWho Makes ChangesAnalysis
n Plan AnalysisDo you want to examine impact upon EFC or Pell
Grant eligibility?
Do you want to analyze dependent and independentstudents separately?
How do you want to define target population?
n Define range(s)
Select ISIR element(s)Tools | Custom Formats | Field Increments
Minimum, Maximum, Increment
8/14/2019 description: tags: Session12
24/37
Session 1224
EFC-Focus: Field RangeReport with CorrectedField Counts
For target population:
n Number of fieldscorrected
n Number of recordsexperiencing an EFCincrease
n Number of recordsexperiencing an EFC
decreasen Number of records
experiencing no EFCchange
By Field Increment
8/14/2019 description: tags: Session12
25/37
Session 1225
Pell-Focus: PellCorrected RecordsReport
For initially Pell eligible amongtarget population:
n Number of records corrected(Selected Fields)
n Number of recordsexperiencing an EFC increase(decrease in Pell)
n Number of recordsexperiencing an EFC decrease(increase in Pell)
n Number of recordsexperiencing no change in Pell
n Summary over and underaward information (limited toinitially Pell eligible)
By Field Increment
8/14/2019 description: tags: Session12
26/37
Session 1226
What to Look for in theField Range and Pell
Corrected Recordsreports
n Where were changes made?
n
Where were changes most associated withEFC increase?
n Where were changes most associated EFCdecrease?
n Where were changes rare or have no effect
on EFC?
8/14/2019 description: tags: Session12
27/37
Session 1227
Questions to followeither Field Range orPell Change reports
Most problematic ranges
n Does your institution explicitly target this
range for verification?n If not specifically targeted do you verify the
majority of this range anyway?
n Activity #2 provides detailed instructions for
computing the percentage of a problematicrange your institution currently verifies
8/14/2019 description: tags: Session12
28/37
Session 1228
Questions to followeither Field Range orPell Change reports
Least problematic ranges
n Would it be prudent to exclude (if
permissible) these ranges fromverification?
n Activity #2 provides detailed instruction forevaluating the implication of excluding lessproblematic ranges
8/14/2019 description: tags: Session12
29/37
Session 1229
How Can Verification
Profiles Be Assessed?n Assess the ability of specific institutional
profiles to capture substantial changes to
eligibility for need-based aidn Assess the tendency of specific institutional
profiles to select applicants with little of noimpact upon eligibility for need-based aid
n Determine which profiles to fine tune ordrop (if permissible)
8/14/2019 description: tags: Session12
30/37
Session 1230
Steps to take before
Assessing Profilesn Plan Analysis
Do you want to examine impact upon EFC?
What is a substantial change for you?
n Define new Queries
Verified and no change to EFC
Verified and substantial change to EFC
8/14/2019 description: tags: Session12
31/37
Session 1231
Three versions of
List VerificationSelection Criteria byCode
Selection Criteria1. Verified
2. Verified and
Substantial Change
3. Verified and No
Change
Numbers generated1. Total
2. Substantial Change
3. No Change
8/14/2019 description: tags: Session12
32/37
Session 1232
Calculate Substantialand No ChangePercentages
For each profile
n Substantial / Total * 100
(79/202 * 100) = 39 percent
n No Change / Total * 100
(115/202 *100) = 57 percent
8/14/2019 description: tags: Session12
33/37
Session 1233
Relate Percentages to
Practicen Be reluctant to alter profiles selecting a
high percentage of substantial changes
n
Be eager (if permissible) to narrow thefocus of profiles selecting a highpercentage of no changes
n Follow up analysis
Which ISIR fields changes?Who makes changes?
8/14/2019 description: tags: Session12
34/37
Session 1234
Tool Enhancements
for 2003-04n Name change from Quality Analysis Tool to
ISIR Analysis Tool.
n Incorporate the use of the VerificationTracking Flag into reports and queryfunction.
n
Allow prior year setup and user definedqueries to be carried forward from 02-3.
8/14/2019 description: tags: Session12
35/37
Session 1235
Tool Enhancements
Continuedn Include summary data on verification
reports that provides detail on the
verification tracking flags applied to therecords in the database.
n Names of existing reports have beenchanged to make them intuitive to the user.
n Sampling? TBD.n New release date May 2003.
8/14/2019 description: tags: Session12
36/37
Session 1236
Related Sessions
n Session #9 CPS Edits, VerificationSelection, and FSAs Quality Analysis Tool.
n #40 Take the Quality Analysis Tool for aTest Drive. (Hands On Session)
n Quality Assurance Focus Group for New
QA Schools or New QA Coordinators atveteran QA SchoolsPC Lab.
8/14/2019 description: tags: Session12
37/37
Session 1237
Thank You for Coming
We appreciate your feedback andcomments. We can be reached:
Email: RhodesD@Calib.com
Francine.Reeves@ed.gov