PowerPoint Presentation

WEBINAR FOR ACBSPFIRST-TIME EVALUATORSPresented Spring 2013

ACCREDITATION COUNCIL FOR BUSINESS SCHOOLS AND PROGRAMSPurpose of this webinar:Provide additional information for our first time evaluators.Provide additional information on how to write strengths and opportunities for improvement comments.Provide an opportunity for you ask any questions you may have.

Agenda:Welcome and IntroductionsRole of the evaluatorResources available to site visit teamScheduling of Consensus Calls and your role during the callTravel arrangements and travel policiesCommunication with institutional contactSite Visit ScheduleFormat for final comments to be included in the feedback reportReview and submission of feedback reportSubmission of expense reports and other forms Any other topics

3ACBSP STAFF CONTACTS:Steve ParscaleDirector of Accreditation913-339-9356, Mobile: 913-526-5285sparscale@acbsp.org

Diana HallerudAssistant Director of Accreditation913-339-9356, Mobile: 816-699-9964dianahallerud@acbsp.org

Attention to telephone

Cell phoneACBSP STAFF CONTACTS:Ron DeYoung, Executive Liaison for the Baccalaureate/Graduate Degree Board of Commissioners, DEYOUNG@nwmissouri.edu Mobile: 816-695-9585

Larry Zachrich, Executive Liaison for the Associate Degree Board of Commisisoners, lzachrich@acbsp.org Mobile: 419-966-7091

5ACBSP STAFF CONTACTS:Maria Hallerud, Accreditation Manager, mhallerud@acbsp.org Office: 913-339-9356

Andrea (Andy) Ernst, Accreditation Administrative Assistant aernst@acbsp.org Office: 913-339-9356

6Primary Role of the Evaluator:Evaluate and validate the information provided in the self-study to the standards and criteria.

Provide feedback ready comments to the business unit regarding strengths and opportunities for improvement based on information from the self-study and observations during the site visit.7Travel arrangements:Review the Travel GuidelinesEvaluators should plan on using their personal credit cards to pay for airfare and be reimbursed through ACBSP. There are exceptions for international travel, when the institution may choose to make the flight arrangements for you or when you can use the ACBSP travel agent. You should always check with Steve Parscale or Diana Hallerud before contacting the ACBSP travel agent.The institution will typically make the hotel reservation for the site visit team, but it varies by institution for the billing arrangements. The institution sometimes sets up a direct billing arrangement. Most institutions ask the site visit team to pay for the hotel room directly and seek reimbursement through ACBSP.Travel expenses are processed on the 15th and the 30th of the month. Submit your expense forms and receipts to Mary Riley via email at maryriley@acbsp.org, fax to 913-339-6226, or send snail mail.

Resources available to you, cont.Files on the CD and notebook you receivedUnder Tab One:Evaluator Training PowerPoint Slides June 2011Scoring Guidelines & WorksheetPractical Guide for Evaluators (includes)ACBSP Observation CardACBSP Site Visit NotesDos & Donts for EvaluatorsSite visit Issue WorksheetSite Visit QuestionsSite Visit Issue Strategy

Resources available to you, cont.Files on the CD and notebook you receivedUnder Tab Two:Process ManualsUnder Tab Three:Standards and CriteriaUnder Tab Four:Evaluator WorkbookFeedback ReportExecutive Summary Template

Resources available to you, cont.Files on the CD and notebook you receivedUnder Tab Five:Travel Reimbursement GuidelinesTravel Reimbursement FormTravel Expense FormEvaluator Agreement FormEvaluator Critique Form

Review Process:Review contents of package and verify that you have received items checked on the checklist.

Read the entire self-study, including the overview, prior to the consensus call. The overview will provide important information regarding mission, vision, student demographics, strategic challenges, etc.

12Reviewing the self-study, cont.READING THE ENTIRE SELF-STUDY DOCUMENT IS IMPORTANT BECAUSEa standard or specific criteria may not be addressed where you expect to see it. Dont be surprised if you see student satisfaction results in Standard 4 rather than Standard 3 or Standard 4 items such as learning outcomes in Standard 3 or Standard 6. the standards have many linkages and information can be found throughout the document. Some self-studies even have most of the results in the appendices rather than the self-study document. the information may not be in the correct place, but the business unit can still meet the standard. It just requires additional effort on your part to locate the information. Using post-its or different colored highlighters when you find information referencing the standards you are responsible for can be helpful when you begin preparing your comments. Review Process:Use the evaluator workbook and begin writing observations prior to the consensus call.

The team chair will notify each team member regarding the standards you will have primary responsibility for reviewing and providing strengths and OFIs. The chair may ask you if you have a preference based on your expertise.14Contact with the Institutional Coordinator/ContactAll communication with the institutional coordinator or contact should be routed through your team chair. Any requests for clarification or additional information should be forwarded to the team chair.This way the institutional coordinator is not responding to duplicate requests for information. Your cooperation is appreciated.The Consensus Call:Using the scoring worksheet, evaluate each of the standards using the best in class to major improvements needed criteria.

The team chair will review site visit logistics and lead the discussion regarding observations after reviewing the self-study.16The Consensus Call continued.For baccalaureate/graduate degree site visits, the commissioner and the mentor are invited to participate in the call and answer questions.

For associate degree site visits, the mentor is invited to participate in the consensus call.17The Consensus Call continued.The team will identify site visit issues and additional information needed to be supplied prior to or during the site visit.

Discussion regarding meetings and the site visit agenda will be finalized.18Overall Scoring Summary WorksheetUse these scoring dimensions to complete the table on the following page. Your individual review will be discussed during the consensus call.Individual Assessment Dimensions Best in ClassGood to ExcellentGoodMajor Improvements NeededPoor

Scoring BandQualitativeQuantitative

ASSOCIATE DEGREE SCORING SUMMARY WORKSHEETEvaluator Name:School Name:SUMMARY OF EXAMINATION ITEMSIndividual Assessment Dimension1 LEADERSHIP2 STRATEGIC PLANNING3 STUDENT, STAKEHOLDER, AND MARKET FOCUS4 MEASUREMENT, ANALYSIS, AND KNOWLEDGE MANAGEMENT4.1 Student Learning Outcomes Assessment4.2 Program Evaluation4.3 Student Assessment5 FACULTY AND STAFF FOCUS5.1 Human Resource Planning5.2 Faculty Qualifications5.3 Faculty Composition5.4 Faculty Deployment5.5 Faculty Load5.6 Faculty Evaluation5.7 Faculty and Staff Professional Development & Scholarly Activities5.8 Faculty Instructional Development5.9 Faculty Operational Policies, Procedures, & Practices6 PROCESS MANAGEMENT6.1 Curriculum6.2 Professional Component6.3 General Education Component 6.4 Business Major Component6.5 Off-Campus Operations and Unique Items6.6 Minimum Grade Requirement6.7 Learning and Academic Resources6.8 Support Services6.9 Educational Innovation6.10 Articulation and Transfer RelationshipsScoring Summary worksheetBaccalaureate/Graduate Degree:

Typical Site Visit Schedule:Review the typical schedule outlined in the Practical Guide for Evaluators.Typically, team members arrive Sunday afternoon/early evening and have a team meeting to discuss site visit issues and finalize the schedule.Monday is typically filled with meetings with administrators, faculty, students, advisory boards, assessment officer, etc.Most teams complete all of the meetings on Tuesday morning and hold any follow-up meetings early afternoon. Tuesday afternoon and evening will be used to finish writing your portion of the report.Your completed portion of the report should be given to the team chair to be used to prepare the executive summary and as a part of the exit interview.CONTINUOUS IMPROVEMENT PROCESSAPPROACHDEPLOYMENT OR IMPLEMENTATION RESULTSandLEARNING

IMPROVEMENT

24Keep in mind:When describing processes for the business school or programs, that the overall value of the results and improvements taking place is dependent on the approach (what they do) and the level and extent of deployment (how they do it). Review Process after Consensus Call:Continue to refine comments for the feedback report prior to the site visit share with the team chair.Ask for guidance when needed there are no dumb questions and every chair has experienced a first visit he or she will welcome any questions you have. 26MOST IMPORTANT THING TO REMEMBER AS AN EVALUATOR:A site visit that is conducted in an organized and professional manner followed by a feedback report that is professionally and clearly written will create a positive impression of what ACBSP represents as an accrediting body.While you may travel to a location new to you, participating in a site visit is not a vacation and should not be considered as such.AS AN EVALUATOR, DORemember that you serve as an ambassador for ACBSP. Most of the faculty and staff you will be interacting with may not have an opportunity to attend regional or national conferences. The site visit experience will be what they base their perception of ACBSP on. Make it a positive one! Come prepared for the site visit with materials read in advance. You are part of a team with your team members depending on you. You will work hard during the site visit and will find it immensely rewarding.Say thank you and complement the institution for their hospitality and availability of materials. Many institutions provide resources and snacks over and above what is expected. Make every attempt to stay on the established schedule and let the next person know in advance if the team is falling behind schedule.Let the team chair contact the institutional contact if additional information or resource is needed. A little diligence on your part will result in a stronger report and assist the Board of Commissioners in making the accreditation decision.AS AN EVALUATOR, DOLet the team chair serve as the primary point of contact in making the arrangements both in advance and during the site visit.Contact the ACBSP office if you encounter any problems and are not sure how to proceed.Remember that each institution has its own mission and operating processes and procedures. Youre only concerned with how the processes and procedures used serve the institution, not how they compare to your institution. Remember that the size and location of the institution you are visiting will determine the type of lodging available during your stay. Institutions are asked to provide reasonable, clean, comfortable accommodations as close to the campus as possible.

AS AN EVALUATOR, DO NOTMarket your services as a consultant or as a potential recruit at the institution. The evaluators role is to evaluate and validate the information in the self-study, not to offer personal assistance when a gap in a particular area is observed or to look for employment opportunities while on campus. Under any circumstances, attempt to recruit faculty as either a student or employee from the institution you are visiting. Even if a faculty member voices an interest in leaving the institution and there is a position available at your institution, do not discuss this opening with the faculty member. Faculty members who express an interest in seeking additional degrees should be directed to ACBSP for a list of institutions available rather than to your specific institution.Cancel appointments with administrators unless you can provide a valid reason for the cancellation. Usually the President or Vice-Presidents work their schedules around the site visit schedule and do not appreciate a sudden cancellation.AS AN EVALUATOR, DO NOT Under any circumstances, make negative comments regarding the business faculty or administration to anyone during the site visit.Criticize the content or quality of the self-study during the site visit. If a standard is not adequately addressed, let the institutional contact know what is missing and ask if the information can be derived from other sources. Wait more than 10 days to submit the feedback report to ACBSP. Do not submit the report unless you proofread the report thoroughly before submitting to ACBSP. All team members should have the opportunity to read the final report prior to submitting the final report. Please make sure the comments are complete and accurately reflect your observations as an evaluator.

Writing Comments:THINGS TO REMEMBER:Comments are based on the requirements of the standards and criteria, not personal opinions or how processes are followed at other institutions.Comments should be clear and concise and should address the situation, how the business unit does (for strengths) or does not meet (for OFIs) the standard, and explain the significance of the situation.Comments should not be prescriptive unless you are using language directly from the standards and criteria.Comments should be written in complete sentences and should use proper grammar and punctuation.Avoid the use of slang or local expressions others may not be familiar with, especially when on international visits. Example: pig in a poke.Writing a strength:If the institution has a strong strategic plan that includes both short and long-term goals andhas a good approach,has been deployed or implemented, demonstrates learning, and identifies improvements, your comment could state:The John Doe School of Business has a strategic plan that has identified both short and long-term goals. The plan is systematic with an approach that has been implemented for all disciplines, provides examples of learning and evidence of improvements the school has made. The strategic plan helps the school to plan for increasing enrollment and the replacement of faculty members as they retire.Writing a strength:If the institution has an assessment instrument and provides evidence of a systematic process and examples of how the results are used to make improvements, you could provide a comment such as:The John Doe School of Business has administered the MFT test for all disciplines for five years. Each department documents their review of the pre and post results each semester during department meetings and makes improvements based on the results. For example, the accounting department added additional projects for preparing financial statements in the ACCT 3300 course.

Writing an opportunity for improvement:

Suppose the institution has identified:increasing enrollment and a large number of faculty scheduled for retirement as strategic challenges. All the strategic objectives in the strategic plan are for one to three years, and faculty are not consulted in the development of the objectives.The opportunity for improvement could read, While the John Doe School of business has strategic objectives for up to three years, there was no evidence of planning beyond three years or involvement by faculty in the development of the objectives. Developing long term goals and involving more stakeholders such as faculty could provide resource planning to meet the increasing enrollment and replacement of faculty members scheduled to retire in the next five years.Writing an Opportunity for Improvement:Suppose the institution administers the Major Field Test (MFT) to all students as a pre and post test. There is no evidence, either in the self-study or on-site, that the results are reviewed to indicate that there is learning or subsequent improvements are made. In addition, the school does not use the data for comparative purposesYour opportunity for improvement comment could read:The John Doe School of Business provided three years of data for the Major Field Test (MFT). However, there is no evidence that the school is using this data to measure how their students are performing in each discipline and how the results compare to other institutions with a similar mission. Analyzing the results to identify program strengths and weaknesses could allow the school of business to make improvements where needed and provide an external source of comparative data.Submitting your final comments:Using your initial review, you will conduct interviews and review documentation to validate the information provided in the self-study.You may find that some initial strengths become OFIs and some OFIs will either be reduced and/or become a strength.Wording for OFIs should change from it is not clear to there is no evidence following the site visit. Before submitting your comments to the chair, read them from the perspective of someone who did not write the self-study. Based on the comment, would the person understand the jist of the comment? Can they relate it to the standard or criteria you are referencing?Ask your team members for help if you get stuck. They wont mind and will be asking you to read a comment for clarity as well.Final Steps:On Tuesday evening, you will submit your final comments to the chair and confer with the team if you have questions.Wednesday morning, you will participate in the exit interview and return home.The chair should send the report for a final review to team members before submitting it to ACBSP. Respond with your comments promptly and let the chair know if you saw evidence of meeting a standard that a team member may have missed or if a comment is not clear to you.Submit your expense report, receipts and evaluation form to ACBSP as soon as you return from the trip.

Questions?While you will work hard, most evaluators find the site visit experience to be very rewarding and no matter how many visits they have participated in, they always learn something new that they can use to improve the processes at their own institution.

Thank you for participating in this webinar. If you have any suggestions for improvement, please feel free to contact me by phone or via email at dianahallerud@acbsp.org.

We hope you enjoy your first site-visit experience.

Sheet1DimensionsApproach/DeploymentDimensionsResultsPoor*no systematic approach evident; anecdotal informationPoor*no results or poor results in areas reportedMajor Improvements Needed*beginning of a systematic approach to the primary purposes of the itemMajor Improvements Needed*early stages of developing trends; some improvements and/or early good performance levels in a few areas*early stages of a transition from reacting to problems to a general improvement orientation*results not reported for many to most areas of impor- tance to the applicants key business requirements*major gaps exist in deployment that would inhibit prog- ress in achieving the primary purposes of the itemGood*a sound, systematic approach, responsive to the pri- mary purposes of the itemGood*improvement trends and/or good performance levels reported for many to most areas of importance to the applicants key business requirements*a fact-based improvement process in place in keyareas; more emphasis is placed on improvement than on reaction to problems*no pattern of adverse trends and/or poor performance levels in areas of importance to the applicants key*no major gaps in deployment, though some areas orbusiness requirementswork units may be in very early stages of deployment*some trends and/or current performance levels -- evaluated against relevant comparisons and/or good to very good relative performance levelsGood to Excellent*a sound, systematic approach, responsive to the over- all purposes of the itemGood to Excellent*current performance is good to excellent in most areas of importance to the applicants key business requirements*a fact-based improvement process is a key manage- ment tool; clear evidence of refinement and improved*most improvement trends and/or performance levels are sustainedintegration as a result of improvement cycles and anal- ysis*many to most trends and/or current performance levels -- evaluated against relevant comparisons and/or bench-*approach is well-deployed, with no major gaps; deploy- ment may vary in some areas or work unitsmarks -- show areas of leadership and very good relative performance levelsBest in Class*a sound, systematic approach, fully responsive to all the requirements of the itemBest in Class*current performance is excellent in most areas of importance to the applicants key business require-*a very strong, fact-based improvement process is amentskey management tool; strong refinement and integra- tion -- backed by excellent analysis*excellent improvement trends and/or sustained excellent performance levels in most areas*approach is fully deployed without any significant weaknesses or gaps in any areas or work units*strong evidence of industry and benchmark leadership demonstrated in many areas

Sheet2

Sheet3

man-serviceExaminer Name:Institution Name:SUMMARY OF EXAMINATION ITEMSINDIVIDUAL ASSESSMENT DIMENSION1 LEADERSHIP1.1 Leadership System2 STRATEGIC PLANNING2.1 Strategic Development Process3 STUDENT AND STAKEHOLDER FOCUS3.1 Student Knowledge and Stakeholder Relationship Enhancement3.2 Student and Stakeholder Satisfaction Determination3.3 Student and Stakeholder Satisfaction Results4 INFORMATION AND ANALYSIS4.1 Selection and Use of Information and Data4.2 Selection and Use of Comparative Information Data4.3 Selection and Use of Information Results5 HUMAN RESOURCE DEVELOPMENT AND MANAGEMENT5.1 Human Resource Planning5.2 Employment5.3 Faculty Deployment5.4 Faculty Size and Load5.5 Faculty Evaluation5.6 Faculty and Staff Development5.7 Faculty Operational Procedures, Policies and Practices5.8 Scholarly and Professional Activities6 EDUCATIONAL AND BUSINESS PROCESS MANAGEMENT6.1 Education Design and Delivery6.2 Management of Educational Support Service Processes and Business Operations6.3 Enrollment Management

&C&"Arial,Bold"&18Scoring Summary Worksheet

pub.sectorExaminer Name:Applicant Name:Applicant No.SUMMARY OF EXAMINATION ITEMSTotal Points Possible APercent Score 0-100% (10% units) BScore(A x B) C1 LEADERSHIP 110 Possible Points1.1 Leadership System800.01.2 Organization Responsibility and Citizenship300.0Category Total1100.02 STRATEGIC PLANNING 80 Possible Points2.1 Strategy Development Process400.02.2 Organization Strategy400.0Category Total800.03 CUSTOMER AND MARKET FOCUS 80 Possible Points3.1 Customer Knowledge400.03.2 Customer Satisfaction and Relationship Enhancement400.0Category Total800.04 INFORMATION AND ANALYSIS 80 Possible Points4.1 Selection and Use of Information and Data250.04.2 Selection and Use of Comparative Information and Data150.04.3 Analysis and Review of Organization Performance400.0Category Total800.05 HUMAN RESOURCE FOCUS 100 Possible Points5.1 Work Systems400.05.2 Employee Education, Training, and Development300.05.3 Employee Well-Being and Satisfaction300.0Category Total1000.06 PROCESS MANAGEMENT 100 Possible Points6.1 Management of Product, Program and Service Processes600.06.2 Management of Support Processes200.06.3 Management of Supplier and Partnering Processes200.0Category Total1000.07 BUSINESS RESULTS 450 Possible Points7.1 Customer Satisfaction Results1250.07.2 Financial Results1250.07.3 Human Resource Results500.07.4 Supplier and Partner Results250.07.5 Organization-Specific Results1250.0Category Total4500.0GRAND TOTAL(D)10000.0D

&C&"Arial,Bold"&18Scoring Summary Worksheet

health careExaminer Name:Applicant Name:Applicant No.SUMMARY OF EXAMINATION ITEMSTotal Points Possible APercent Score 0-100% (10% units) BScore(A x B) C1 LEADERSHIP 110 Possible Points1.1 Leadership System800.01.2 Public Responsibility and Citizenship300.0Category Total1100.02 STRATEGIC PLANNING 80 Possible Points2.1 Strategy Development Process400.02.2 Organizational Strategy400.0Category Total800.03 FOCUS ON PATIENTS, OTHER CUSTOMERS AND ON MARKETS 80 Possible Points3.1 Patient/Customer and Health Care Market Knowledge400.03.2 Patient/Customer Satisfaction and Relationship Enhancement400.0Category Total800.04 INFORMATION AND ANALYSIS 80 Possible Points4.1 Selection and Use of Information and Data250.04.2 Selection and Use of Comparative Information and Data150.04.3 Analysis and Review of Organizational Performance400.0Category Total800.05 STAFF FOCUS 100 Possible Points5.1 Work Systems400.05.2 Staff Education, Training, and Development300.05.3 Staff Well-Being and Satisfaction300.0Category Total1000.06 PROCESS MANAGEMENT 100 Possible Points6.1 Design and Delivery of Health Care700.06.2 Management of Support Processes300.0Category Total1000.07 ORGANIZATIONAL PERFORMANCE RESULTS 450 Possible Points7.1 Patient/Customer Satisfaction Results1000.07.2 Health Care Results1200.07.3 Financial and Market Results800.07.4 Staff and Work System Results500.07.5 Organization-Specific Results1000.0Category Total4500.0GRAND TOTAL(D)10000.0D

&C&"Arial,Bold"&18Scoring Summary Worksheet

educationExaminer Name:School's Name:SUMMARY OF EXAMINATION ITEMSIndividual Assessment Dimension1 LEADERSHIP2 STRATEGIC PLANNING3 STUDENT AND STAKEHOLDER FOCUS4 MEASUREMENT AND ANALYSIS OF STUDENT LEARNING AND PERFORMANCE4.1 Selection and Use of Information and Data4.2 Selection and Use of Information Results4.3 Selection and Use of Comparative Information and Data4.4 Continuous Process Improvement5 FACULTY AND STAFF FOCUS5.1 Human Resource Planning5.2 Employment5.3 Faculty Deployment5.4 Faculty Size and Load5.5 Faculty Evaluation5.6 Faculty and Staff Development5.7 Faculty Operational Procedures, Policies, and Practices5.8 Scholarly and Professional Activities6 EDUCATIONAL AND BUSINESS PROCESS MANAGEMENT6.1 Education Design and Delivery6.2 Management of Educational Support Service Processes and Business Operation Processes6.3 Enrollment Management

&C&"Arial,Bold"&18Scoring Summary Worksheet

Sheet2

&APage &P

Sheet3

&APage &P

Sheet4

&APage &P

Sheet5

&APage &P

Sheet6

&APage &P

Sheet7

&APage &P

Sheet8

&APage &P

Sheet9

&APage &P

Sheet10

&APage &P

Sheet11

&APage &P

Sheet12

&APage &P

Sheet13

&APage &P

Sheet14

&APage &P

Sheet15

&APage &P

Sheet16

&APage &P