Home >Documents >Welcome! If you joined the conference via telephone, please enter your audio pin if you havent...

Welcome! If you joined the conference via telephone, please enter your audio pin if you havent...

Date post:27-Mar-2015
View:215 times
Download:1 times
Share this document with a friend
  • Slide 1

Welcome! If you joined the conference via telephone, please enter your audio pin if you havent already. (This is an example, your audio pin will be different) If you joined with a microphone and headset or speakers (VoIP), please be sure your device volume settings are properly adjusted. If your mic & headset is not working, please try unplugging and re-plugging in your device. If you have any questions, please enter them in the questions module. We will begin promptly at the top of the hour. Slide 2 DOC110: Evaluating Document Assembly Projects September 26, 2008 Slide 3 GoToWebinar Interface Maximize/minimize your screen with the chevron symbol Telephone participants need to enter their audio pin Please share your questions and knowledge! Ask a question in the questions log Raise your hand Slide 4 Panelists Kate Bladow, Pro Bono Net Steve Rapp, Idaho Legal Aid Services Bristow Hardin, Legal Services Corp. Slide 5 Todays Training What is evaluation? Why evaluate? How do you evaluate a program? Planning for an evaluation The process Case study: Idaho Legal Aid Services What is LSC looking for in evaluation plans and reports? Slide 6 Evaluation Defined Systematically collecting data to answer questions of interest. Slide 7 Reasons to Evaluate a Program To find out if it was successful To build buy-in To document lessons learned and potential best practices To identify areas for improvement To justify the cost and effort Slide 8 Evaluation Process 1. Draft an evaluation plan. 2. Design and test data collection instruments. 3. Collect baseline data. 4. Collect additional data. 5. Analyze data and report findings. 6. Integrate necessary changes. 7. Repeat steps. Slide 9 Draft an Evaluation Plan Slide 10 Example Evaluation Plan Project Goal: Use A2J author and HotDocs to create guided interviews and court form templates that improve the ability of low income people to easily, effectively and efficiently produce and file court forms without the assistance of an attorney. Project ObjectivesStrategies/ActivitiesEvaluation QuestionsEvaluation Data Objective 1 Use A2J Author and HotDocs to create guided interviews and court form templates that enable pro se litigants to easily produce accurate pro se pleadings. Produce [insert number] guided interviews and [insert number] interactive court forms. Identify forms/interviews to automate Hire forms developer to help develop guided interviews and automated forms Work with courts to ensure that interviews/forms are accurate, consistent with court requirements and will be acceptable to the courts. Conduct usability testing of templates to ensure users (clients and/or advocates) can easily understand and use the interviews/forms. (Note: forms should be at the eight-grade reading level of lower.) Modify templates to address any problems identified in usability testing Publish forms/interviews on statewide website What forms/interviews were selected for automation? Was developer hired to develop guided interviews and automated forms? What interviews were produced? What forms were automated? Have courts confirmed that the forms are accurate, consistent with court requirements and acceptable for use? What usability testing was conducted? What were the results of the usability testing? What changes were made to the automated forms based on the initial testing? What automated forms and guided interviews been published on the statewide website? To what extent have the automated forms and guided interviews made it easier for self- represented litigants to prepare accurate court pleadings? Confirmation that developer hired. List of interviews and automated court forms produced. Court confirmation that forms are forms are accurate, consistent with court requirements and acceptable for use Description of usability testing Results of usability testing Description of notable changes made to forms and interviews following initial testing List of forms/interviews published on statewide website User interviews/surveys with: Users Court staff User feedback from the website Field observation data Objective 2 Increase the availability and use by low-income persons of automated forms Partner with courts and other stakeholders to develop and implement outreach initiatives to: foster courts acceptance of the forms maximize clients use of forms Partner with public libraries, courthouses and other groups to increase access points where clients can: access and complete forms receive assistance in completing forms What automated forms and guided interviews have been posted the statewide website? Was an outreach plan developed? What outreach activities were conducted? To what extent are the different forms being used by clients? How many judicial districts/courts have accepted the automated forms? How many new access points have been established where clients can access the forms? At how many access points can clients receive assistance in completing the forms? Copy of outreach plan Description of outreach activities Statistics from national server on number of times different forms have been accessed and/or downloaded List of judicial districts that have accepted the automated forms List of new access points (with description of services available to clients) Slide 11 Design and test instruments Time Trials Focus Groups Usage Statistics Usability Tests Document Comparisons Interviews Surveys Slide 12 Collect baseline data Allows you to compare before and after. Can be collected while the document assembly content is being developed. Keep this data collection consistent with your later data collection. Slide 13 Collect additional data Happens after implementation. Stay consistent with your early data collection. Slide 14 Analyze data and report findings Project Goals and Objectives Evaluation Data and Methodologies Summary of Major Accomplishments Recommendations and Future Steps In-Depth Analysis of Accomplishments Factors affecting project accomplishments Strategies to address major challenges Major lessons and recommendations Slide 15 Integrate necessary changes Use the feedback that you gathered. Tweak the program and then collect more evaluation data to find out if it really was a positive change. Slide 16 Repeat Plan Design CollectAnalyze Report Integrate Slide 17 Case Study Idaho Legal Aid Services Slide 18 Idaho Project Highlights Partnership between Idaho Legal Aid Services and the Idaho Supreme Court Funding provided by the Legal Services Corporation and the State Justice Institute Successful good feedback and good filings 2008 National Center for State Courts Slide 19 Slide 20 Slide 21 Slide 22 Slide 23 Slide 24 A Conversation with LSC Slide 25 Questions? Kate Bladow [email protected] Steve Rapp - [email protected] Bristow Hardin [email protected] Slide 26 Resources Slide 27 DA and Legal Aid Specific Document Assembly Support http://www.probono.net/dasupport http://www.probono.net/dasupport LSC Evaluation Plan and Report Forms http://tig.lsc.gov/eval_nonwebsite_grants.php http://tig.lsc.gov/eval_nonwebsite_grants.php SelfHelpSupport.org http://bit.ly/y4c4s http://bit.ly/y4c4s Slide 28 Overview and Theory W.K.K. Foundation Evaluation Handbook http://bit.ly/1fg3n4 http://bit.ly/1fg3n4 Outcome Measurement Resource Network http://bit.ly/4rD2eY http://bit.ly/4rD2eY Basic Guide to Program Evaluation http://bit.ly/eHHjW http://bit.ly/eHHjW Intelligent Measurement Blog http://intelligentmeasurement.wordpress.com/ http://intelligentmeasurement.wordpress.com/ Point K http://bit.ly/HQL2k http://bit.ly/HQL2k Slide 29 Surveys LSNTAP Survey Tool http://bit.ly/2ARmCF http://bit.ly/2ARmCF Creating Effective Online Surveys http://bit.ly/1bLpwg http://bit.ly/1bLpwg Likert Scales http://bit.ly/4nlZlh http://bit.ly/4nlZlh Slide 30 Usability Testing Usability Methods http://usability.gov/methods/ http://usability.gov/methods/ Assess Your Sites Navigation & Structure http://bit.ly/1Bc1Pv http://bit.ly/1Bc1Pv Useit.com http://www.useit.com/ http://www.useit.com/ Slide 31 Focus Groups How to Use Focus Groups http://bit.ly/NHLBA http://bit.ly/NHLBA Slide 32 Data Analysis Excel Skills for Advocates http://www.lsntap.org/OFF105 http://www.lsntap.org/OFF105 W.K.K Foundation Data Analysis http://bit.ly/44nU2h http://bit.ly/44nU2h How to Analyze Data http://bit.ly/3sRnvP http://bit.ly/3sRnvP Overview Introduction to Statistics http://bit.ly/3TpaMd http://bit.ly/3TpaMd The Complete Idiot's Guide to Statistics http://bit.ly/2t0Hen or http://bit.ly/4hRQok http://bit.ly/2t0Henhttp://bit.ly/4hRQok Slide 33 Attributions Image Slide 6 SMercury98 http://www.flickr.com/photos/smercury98/ http://www.flickr.com/photos/smercury98/ Image Slide 8 markymarko http://www.flickr.com/photos/cybercafe/ http://www.flickr.com/photos/cybercafe/ Image Slide 10 roboppy http://www.flickr.com/photos/roboppy/ http://www.flickr.com/photos/roboppy/ Image Slide 10 gergenzola http://www.flickr.com/photos/gergenzola/ http://www.flickr.com/photos/gergenzola/ Image Slide 10 Eric Spiegelman http://www.flickr.com/photos/ericspiegelman/ http://www.flickr.com/photos/ericspiegelman/ Image Slide 10 jimmedia http://www.flickr.com/photos/jimmediaart/ http://www.flickr.com/photos/jimmediaart/ Image Slide 10 cambodia4kids http://www.flickr.com/photos/cambodia4kidsorg/ http://www.flickr.com/photos/cambodia4kidsorg/ Image Slide 10 ifindkarma http://www.flickr.com/photos/ifindkarma/ http://www.flickr.com/photos/ifindkarma/ Images Slide 13 johnbullas http://www.flickr.com/photos/johnbullas/ Image Slide 27 Faeryan http://www.flickr.com/photos/faeryan/ http://www.flickr.com/photos/faeryan/ Content Slides 7, 8, 10-13, 16-18 Adapted from Court Leadership and Self-Represented Litigation Package: Module 7 http://bit.ly/3qjLJ4 http://bit.ly/3qjLJ4

Popular Tags:

Click here to load reader

Embed Size (px)