View our IACBE Public Disclosure of Student Learning Form ...

IACBE Annual Report: 2015-2016 1

Saint Francis University School of Business Outcomes Assessment Results For Academic Year: 2015-2016

Section I: Student Learning Assessment

Student Learning Assessment for Saint Francis University Undergraduate Business Programs:

Intended Student Learning Outcomes for Saint Francis University Undergraduate Business Programs:

1. In the Spirit of Saint Francis in the Marketplace, students will be able to demonstrate knowledge of the ethical value choices and responsibilities of businesses and business leaders so that they will be able to engage in ethical decision making within a Franciscan context.

2. Students will be able to demonstrate knowledge of the fundamental principles in business in the functional areas of accounting, information systems, finance, management, and marketing.

3. Students will be able to demonstrate knowledge of the legal, social, and economic environments of business.

4. Students will be able to demonstrate knowledge of the global dimension of business.

5. Students will be able to demonstrate critical thinking by demonstrating their ability to use financial statements and other decision-support tools in the process of effective decision making.

6. Students will be able to demonstrate the ability to communicate effectively and professionally.


Assessment Tools for Intended Student Learning Outcomes— Direct Measures of Student Learning: Performance Targets/Criteria for Direct Measures:

1. End-of-Program Comprehensive Exam (CBE 4.0 from The Technology Fluency Institute)

1. The percentage of SFU students scoring in the basic, proficient, or mastery level will be higher than the national average on the following exam domains: accounting, management, finance, and marketing.

2. The percentage of SFU students scoring in the basic, proficient, or mastery level will be higher than the national average on the following exam domains: accounting, management, finance, and marketing.

3. The percentage of SFU students scoring in the basic, proficient, or mastery level will be higher than the national average on the following exam domains: economic environment, legal environment, and social environment.

4. The percentage of SFU students scoring in the basic, proficient, or mastery level will be higher than the national average on the International Business domain.

2. End-of-Program Comprehensive Business Simulation (Glo-Bus) and Capstone Course

1. At least 50% of SFU students will be at or above the national median on the learning assurance report generated from the Glo-Bus Simulation for the measures of Financial Analysis, Financial Management, Marketing Management, Strategic Analysis and Planning, and Corporate Social Responsibility.


Assessment Tools for Intended Student Learning Outcomes— Indirect Measures of Student Learning: Performance Targets/Criteria for Indirect Measures:

1. IDEA Course Evaluation Analysis 1. During the fall and spring semesters, over 75% of business courses will be above the national average on the measure “Progress on relative objectives” of the IDEA survey.

2. Senior Exit Survey 1. In Question 7 of the Senior Exit Survey, 90% of students will rate their knowledge of ethical values choices and the ethical responsibility of business persons as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level

2. In Question 7 of the Senior Exit Survey, 90% of students will rate their knowledge of the fundamental business principles in various domains such as accounting, management, marketing, finance, and information systems as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level

3. In Question 7 of the Senior Exit Survey, 90% of students will rate their knowledge of the legal, social, and economic dimension of business as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level

4. In Question 7 of the Senior Exit Survey, 90% of students will rate their knowledge of the global dimension of business as as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level

5. In Question 6 of the Senior Exit Survey, 75% of students will rate the effectives of their business education in giving them ability to use financial statement and other decision tools as as "Effective" or "Highly Effective"

6. In Question 6 of the Senior Exit Survey, 75% of students will rate the effectives of their business education in providing


them with the skills and abilities necessary for professional and effective communication as "Effective" or "Highly Effective"

Summary of Results from Implementing Direct Measures of Student Learning: Performance Target Was…

Met Not Met

1. End-of-Program Comprehensive Exam (CBE 4.0 from The Technology Fluency Institute)

After excluding traditional tranfers students, 53 graduating seniors completed the end-of-program Comprehensive Business Exam. The results below represent the percentage of SFU graduating seniors that scored in the mastery, basic, or proficient levels for the various domains as compared to the national averages. Bold domain indicate where SFU fell below target. 2015-2016 (Current Year) Accounting: National: 39.32%, SFU:47.17% (25 of 53) Management: National: 51.19%, SFU:56.60% (30 of 53) Finance:National:33.06%, SFU:47.17% (25 of 53) Marketing: National: 61.78%, SFU:50.94% (27 of 53) Economic Envioronment: 48.21%, SFU:64.15% (34 of 53) Legal Environment: National: 44.4%, SFU: 47.17% (25 of 53) Social Environment:National:60.17%, SFU:64.15% (34 of 53) Intl Bus/Global Dimension: National:64.39%, SFU:67.29% (36 of 53)

X X


2014-2015 (Prior Year) Accounting: National: 39.34%, SFU:52.5% Management: National: 54.9%, SFU: 35% (two years in a row below benchmark) Finance:National:33.5%, SFU:37.5% Marketing: National: 65.05%, SFU:60.0% (first year below) Economic Envioronment: 48.32%, SFU:57.5% Legal Environment: National: 44.62%, SFU: 50% Social Environment:National:59.8%, SFU:47.5% (two years in a row below benchmark) Intl Bus/Global Dimension: National:65.8%, SFU:67.5%

2013-2014 (two years prior) Accounting: National: 39.52%, SFU:44.2% Management: National: 56.1%, SFU: 55.8% Finance:National:30.2%, SFU:39.5% Marketing: National: 63.8%, SFU:69.8%" Economic Envioronment: 44.9%, SFU:60.5% Legal Environment: National: 45%, SFU: 41.9% Social Environment:National:59.8%, SFU:53.5% Intl Bus/Global Dimension: National:66.1%, SFU:69.8%


2. End-of-Program Comprehensive Business Simulation (Glo-Bus) and Capstone Course At least 50% of SFU students will be at or above the national median on the learning assurance report generated from the Glo-Bus Simulation for the measures of Financial Analysis, Financial Management, Marketing Management, Strategic Analysis and Planning, and Corporate Social Responsibility. 2015-2016 (Current year) Financial Analysis: 53.13% at or above national median (34 of 64) Financial Management: 84.38% at or above national median (54 of 64) Marketing: 60.94% at or above national median (39 of 64) Corporate Social Responsibility: 68.75% at or above national median (44 of 64)

X

Summary of Results from Implementing Indirect Measures of Student Learning: Performance Target Was…

Met Not Met

1. IDEA Course Evaluation Analysis In the fall 2015 and spring 2016 semesters, 87% and 89% respectively of courses were at or above the national average on the measure “Progress on Relative Objectives”as assessed by the nationally-normed IDEA course evaluation system.

X


2. Senior Exit Survey In the fall 2015 semester, 47 seniors completed the senior exit survey sharing their perceptions of their undergraduate educational experience. As part of this survey, students were asked to rate their knowledge of fundamental principles and business domains. The percentage of students that rated their knowledge of fundamental business principles and concepts in various domains as "Average", "Good" or "Excellent" and at the "Good" or "Excellent" respectively are as follows: 2015-2016 (current year) Ethical value choices and ethical responsibility In business 100%/80.4% Accounting: 95.65%/75.56% Management: 100%/84.8% Marketing: 100%/82.6% Finance: 91.3%/53.2% (below benchmark last year) Info Systems: 97.8%/69.2% Knowledge of legal, social, and economic dimension of businsess: 95.65%/76.2% Global Dimension of Business 97.8%/80.4% As well for the 2015-2016 cohort, 84.78% of students rated the effectiveness of their business education in giving them the ability to use financial statement and other decision tools as as "Effective" or "Highly Effective" and 89.13% rated the effectives of their business education in providing them with the skills and abilities necessary for professional and effective communication as "Effective" or "Highly Effective" meeting our targets for both measures.

X


2014-2015 (prior year) Ethical value choices and ethical responsibility In business 91.4%/74.5% Accounting: 97.9%/72.3% Management: 100%/85% Marketing: 97.9%/80.9% Finance: 82.3%/44.7% (first year below benchmark) Info Systems: 91.5%/70.2% Knowledge of legal, social, and economic dimension of businsess: 95.7%/71.7% Global Dimension of Business 97.9%/70.2% As well for the 2014-2015 cohort, 83% of students rated the effectiveness of their business education in giving them the ability to use financial statement and other decision tools as as "Effective" or "Highly Effective" and 87.2% rated the effectives of their business education in providing them with the skills and abilities necessary for professional and effective communication as "Effective" or "Highly Effective" meeting our targets for both measures. 2013-2014 (two years prior year) Ethical value choices and ethical responsibility In business 95.6%/86.9% Accounting: 89.3%/61.7% Management: 100%/78.7% Marketing: 100%/85% Finance: 80.85%/55.3% Info Systems: 93.4%/63% Knowledge of legal, social, and economic dimension of businsess: 95.6%/76.5% Global Dimension of Business 91.5%/76.5%


As well for the 2013-2014 cohort, 93.6% of students rated the effectiveness of their business education in giving them the ability to use financial statement and other decision tools as as "Effective" or "Highly Effective" and 97.9% rated the effectives of their business education in providing them with the skills and abilities necessary for professional and effective communication as "Effective" or "Highly Effective" meeting our targets for both measures.

Proposed Courses of Action for Improvement in Areas for which Performance Targets Were Not Met:

1.

This is the fifth year we have administered the comprehensive business exam for graduating seniors. Each year, there has been at least one domain where we failed to meet our benchmark. For the fith consecutive year, some of the many performance measures that were applied to the end-of-program comprehensive exam were not met. After this exam was first administered at SFU during the 2011-2012 academic year, we believed it would be too impulsive to take major corrective action on the basis of only one data point. However, after the second year, as we began to see evidence of a pattern, the School of Business formed a sub-committee of the School of Business faculty to study both the exam as a measure, our metrics, our performance, and recommend any necessary corrective action. This resulted in changes refocusing 2 of the management courses in the CPC and it appears that refocusing may have finally paid dividends as for the first time in two years both the management domain and social environment domain which overlaps with management decision making and ethics were above the benchmark. We will continue to monitor this domain in future years. However it appears, that the marketing domain has now missed the benchmark for two consecutive years. As such, just as was done for management , a sub-committee of School of Business members will analyze the marketing measure, our curriculum as pertains to this domain and decide what corrective actions can be taken. Perplexingly, our students seem to perceive that they are gaining a great deal of knowledge within the marketing domain. This year 100% of students in the senior exit survey reported that their knowledge of the fundamental


business principles in marketing were at least “average” and 83% reported their knowledge to be at the “good” or “excellent” level. There is obviously a disconnect between these two measures and further study is needed by our outcomes assessment sub-committee.


MBA Program Outcomes Assessment Results

For Academic Year: 2015-2016 Section I: Student Learning Assessment

Student Learning Assessment for (Program 1)

Intended Student Learning Outcomes for (Program 1) :

1. In the Spirit of Saint Francis in the Marketplace, students will be able to demonstrate in-depth knowledge of the ethical value choices and responsibilities of businesses and business leaders so that they will be able to engage in ethical decision making within a Franciscan context.

2. Students will demonstrate skills in problem recognition and critical decision making applying qualitative and quantitative methods to empirical business situations (case studies will be used to develop problem recognition skills, critical thinking, and decision making at an advanced level to demonstrate a solid analytical thinking competency).

3. Students will display skills in strategic analysis and integration.

4. Students will have the ability to communicate with relevant stakeholders at a proficient (or advanced level) by developing (a) effective written communication skills, and (b) effective oral communication and business presentation skills.

5. Students will develop the ability to effectively work with a team of colleagues on a project.


1. Peregrine Masters Level CPC-Based Comp Exam (Adopted in 2014)

This end-of-the program comprehensive exam is designed to assess student knowledge on 14 CPC topics covered in the MBA program. The desired target is that the aggregate scores for Saint Francis University MBA students to be at or above the satisfactory level for each CPC topic and for the overall test. The minimum acceptable percentage score is 40% and is considered satisfactory.


The MBA Program Faculty will also compare our aggregate student test scores per CPC topic to a benchmark consisting of participating MBA programs in the Middle States Region just for informational purposes during the first three years of using the test. Another performance target is that the aggregate percentile ranking for Saint Francis University will lie at 45th percentile or higher for the whole test. Revision in Fall 2015: This end-of-the program comprehensive exam is designed to assess student knowledge on 14 CPC topics covered in the MBA program. The performance target is that 75% of the Students will achieve a comprehensive test score of 40t% or higher.

2. The Capstone Business Simulation and Comp-XM used in the Strategic Management Capstone course (MBA 550)

This Capstone Simulation in case measures knowledge acquired in the functional business areas, communication and teamwork skills, and critical thinking and problem solving. There is also a capstone exam that students complete individually. The targeted aggregate performance test score in each of six functional business knowledge areas covered by the simulation and the related test is to be at or above the national average in each discipline. Furthermore, the graduating MBA students are expected to successfully complete the simulation and achieve a cumulative cohort test score at the 45th percentile or higher on each of the two assessment elements: Business Acumen (based on simulation results) and Business Knowledge.



1. Annual MBA Exiting Graduates’ Learning Self-Assessment and Satisfaction Survey, including perceptions of their learning outcomes and other programmatic-level outcomes. Revised MBA ISLOs: ISLO 1. Integrate business theory and practice for the purpose of strategic analysis. ISLO 2. Students will demonstrate skills in problem recognition and critical decision making applying qualitative and quantitative methods to empirical business situations. ISLO 3. Students will demonstrate the ability to professionaly communicate. ISLO 4. Students will display skills in strategic analysis and integration. ISLO 5: Students will develop the ability to effectively work with a team of colleagues on a project. ISLO 6. Identify and analyze the ethical obligations and responsibilities of managers and their organizations.

Achieve at least a 4.0/5.0 or higher “mean” Likert scale satisfaction score on five of the key universal measures linked to student learning and satisfaction. A 4.0 indicates “agree” on a Likert scale and a 5.0 indicates “strongly” agree. Revision in Fall 2015: The MBA program Faculty assessed all six revised ISLOs in the Annual MBA Exit Survey, effective Spring 2016. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to integrate business theory and practice for the purpose of strategic analysis to be "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability demonstrate skills in problem recognition and critical decision making skills applying qualitiative and quantitative methods to real life situations as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to to professionally communicate as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to more effectively display skills in strategic analysis and integration as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to more effectively work with a team of colleagues on a project as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level.


In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to identify and analyze ethical obligations and responsibilities of management and their organizations as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level

2. Annual MBA Graduate Career Placement Results Study. Achieve an annual placement rate of 90% or higher for MBA graduates from the prior graduation year, 2015. That implies that they either found a relevant job in their chosen field or attended another graduate school within 6 months of graduation.


Met Not Met

1. Peregrine Masters Level CPC-Based Comp Exam The summary results for 2015 - 2016 (n=43) are: CPC Topic: SFU Average Score: Benchmark Comparison: Variance (Fav/UnFav): Accounting 48.37 47.25 Favorable Business Ethics 50.47 50.02 Favorable Business Finance 51.63 40.69 Favorable CPC Topic: SFU Average Score: Benchmark Comparison: Variance (Fav/UnFav):

X


Business Integration/Strategic Management 52.79 51.34 Favorable Business Leadership 54.42 55.25 Unfavorable Economics 50.00 48.61 Favorable Macroeconomics 49.30 47.83 Favorable Microeconomics 50.70 49.53 Favorable Management 52.79 52.85 On Par Human Resource Mgmt. 58.62 54.70 Favorable Organizational Behavior 54.93 56.10 Unfavorable Marketing 54.65 53.77 Favorable Final Test Score 51.89 48.26 Favorable SFU MBA students performed better than the IACBE Region II (Middle States) Region Benchmark on the overall test score and in 9 of the 12 CPC areas. There were only two areas that SFU student fared worse in: Business Leadership (54.42 vs. 55.25) and Organizational Behavior (54.93 vs. 56.10). It appears that the 2015-2016 MBA student cohort perfomed better than the 2014-2015 group that follows. The summary results for 2014- 2015 (n=50) are: CPC Topic: SFU Average Score: Benchmark Comparison: Variance (Fav/UnFav): Accounting 44.80 47.25 Unfavorable Business Ethics 48.00 50.02 Unfavorable Business Finance 46.20 40.69 Favorable Business Integration/Strategic Management 52.40 51.34 Favorable


CPC Topic: SFU Average Score: Benchmark Comparison: Variance (Fav/UnFav): Business Leadership 52.20 55.25 Unfavorable Economics 43.60 48.61 Unfavorable Macroeconomics 43.60 47.83 Unfavorable Microeconomics 43.60 49.53 Unfavorable Management 46.60 52.85 Unfavorable Human Resource Mgmt. 50.30 54.70 Unfavorable Organizational Behavior 51.81 56.10 Unfavorable Marketing 50.81 53.77 UnFavorable Final Test Score 48.08 48.26 On Par Note: The CPC-Based COMP Exam Score of 40 – 59% is considered to be average or satisfactory. -------------------------------------------------------------------------------------------------------------------------------- 60.5% of the MBA students met or exceeded the Peregrine MBA CPC Exam targeted test score benchmark set in 2015-2016, which is less than the goal of 75% passing it. 26 out of 43 students met or exceeded the benchmark threshold score desired. The School of Business Faculty need to devise a better way of incentizing students to do well on the exam since it is neither part of a course grade or requires a passing score for graduation. We also should closely investigate the exam content to determine it aligns with the curriculum. In other words, the Graduate Business faculty should identify an gaps in curriculum coverage relative to the test content and make any necessary modifications.

X


2. The Capstone Business Simulation and Comp-XM used in the Strategic Management Capstone course (MBA 550) each spring... Comp-XM Exam Results By Functional Domain Area: Functional Domain: SFU Average Score: National Average Score: Variance (Fav/UnFav): Accounting 66% 59% +7% Fav. (3rd year in a row) Finance 64% 59% + 5% Fav. (3rd year in a row) Strategy 67% 60% + 7% Fav. (3rd year in a row) Marketing 73% 58% +15% Fav.(3rd year in a row) Operations 65% 50% +15% Fav. (3rd year in a row) Human Resources 69% 56% +13% Fav (3rd year in a row) Class Median Percentile: Overall Simulation Result for the MBA Graduating Class Cohorts, Fall 2015 MBA Class it was at the 55th Pecentile and and for the Spring 2016 MBA Class it was at the 58th Percentile. (Business Acumen). Our performance target is the 45th or higher and our scored result is favorable. Overall Board Inquiry Result for the MBA Graduating Class Cohorts, Fall 2015 MBA Class achieved the 66th Percentile while the Spring 2016 MBA Class achieved an 82nd Percentile ranking (Business Knowledge). Our target is the 45th percentile and our scored result is very favorable.

X

X X



Met Not Met

1. Annual MBA Exiting Graduates’ Learning Self-Assessment and Satisfaction Survey, including perceptions of their learning outcomes and other programmatic-level outcomes. Select responses and accompanying ratings from the MBA class of 2015 and 2016: As a result of the MBA program, I have gained effective oral communication and business presentation skills: 4.13/5.0 in 2016 and 4.07/5.0 in 2015. As a result of the MBA program, I have the ability to effectively work with a team of colleagues on a project: 4.63/5.0 in 2016 and 4.00 in 2015. MBA professors displayed enthusiasm about their subject matter: 4.67/5.0 in 2016 and 4.57/5.0 in 2015. MBA professors had the ability to clearly explain their subject matter: 4.46/5.0 in 2016 and 4.47/5.0 in 2015. MBA professors devoted time to teaching real world applications of their subject matter: 4.58/5.00 in 2016 and 4.37/5.0 in 2015. I would recommend the MBA program to other students: 4.46/5.0 in 2016 and 4.17/5.0 in 2015. The MBA program staff was developed to providing excellent customer service: 4.50/5.0 in 2016 and 4.60/5.0 in 2015. I would rate the quality of the MBA program to be excellent: 4.25/5.0 in 2016 and 4.03/5.0 in 2015. New Revision in Fall 2015: Student Measure their Progress on the Revised MBA Intended Student Learning Outcomes. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to integrate business theory and practice for the purpose of strategic analysis to be "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Response: 100% on the first target and 96% on the second target.

X

X


In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability demonstrate skills in problem recognition and critical decision making skills applying qualitiative and quantitative methods to real life situations as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Response: 100% regarding both targets. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to to professionally communicate as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Response: 100% on the first target and 79% on the second target. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to more effectively display skills in strategic analysis and integration as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Response: 100% on the first target and 92% on the second target. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to more effectively work with a team of colleagues on a project as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Response: 100% on both targets. In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to identify and analyze ethical obligations and responsibilities of management and their organizations as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Response: 100% on the first target and 92% on the second target.

X

X

X

X

X


2. Annual MBA Graduate Career Placement Results Study. The MBA Class of 2014 Career Placement Rate was 97%, with an 86% response rate (29/35 reporting) The MBA Class of 2015 Career Placement Rate was 91% with an 95% response rate (35/37 reporting). The averaging starting salary was in the range of $40,000 - $49,999.

X


1. Saint Francis University used the Peregrine Masters Level CPC-Based COMP Exam for the first time in January 2014 and were not surprised by the results. There was a gap between actual and desire results. It compares our MBA student performance to those at approximately 33 other MBA programs participating in the assessment program. We did meet the overall target percentile for our MBA students and will monitor the 14 CPC topic areas for performance and trends. Since we have only three years of results, we need to exercise restraint and caution in interpreting too much from them. We also need to work at aligning our curriculum content to the exam (sync the two together a bit). The MBA program faculty will monitor the aggregate CPC topic scores and assess deviations from the average national performance in the respective CPC areas. They will pay particular attention to unfavorable variances occurring in one particular CPC two or three years in a row and make necessary program improvements and changes deemed to be necessary or warranted. However, the 2015-2016 test results appear to be strong and compare favorably with the comparison groups of Middle States Region schools administering the test. Revision in Fall 2015: This end-of-the program comprehensive exam is designed to assess student knowledge on 14 CPC topics covered in the MBA program. The performance target is that 75% of the Students will achieve a comprehensive test score of 40% or higher. 60.5% of the MBA students met or exceeded the Peregrine MBA CPC Exam targeted text score benchmark set in 2015-2016, which is less than the goal of 75% passing it. 26 out of 43 students met or exceeded the benchmark threshold score desired. The School of Business Faculty need to devise a better way of incentizing students to do well on the exam since it is neither part of a course grade or requires a passing score for graduation. We needed six additional students to achieve the target in order to meet


the performance objective this past year. Since this is first year the MBA Faculty are using this new performance target, they will see what occurs next year and consider ways of incentizing students to strive for a higher scores. It is also recommended the Graduate Business faculty identify an gaps in curriculum coverage relative to the test content and make any necessary modifcations necessary.

2. After three years of assessment date using the Capstone Business Simulation and Comp-XM direct learning results that compares our student results to those of other MBA students across the country indicate that Saint Francis University MBA students are performing above the national norms in six disciplines according the individual Business Knowledge Test as well as performing well on the simulation. This Capstone Simulation in the case measures knowledge acquired in the functional business areas, communication and teamwork skills, and critical thinking and problem solving.

3. The MBA Student Exit survey was modified in Fall 2015 to explicitly measure the revised Six Intended Student Learning Outcomes (ISLOS) and the results affirmed that students perceived that they are achieving these learning outcomes above the specified targeted performance levels.

4. While the Business faculty value career placement outcomes, this will be last year they are included in our indirect measure of student learning. They are appropriately measured under our operational assessment measures and not as indirect measures of learning.

5. Demonstrate stronger evidence in future reporting that these assessment results are being used by graduate faculty in a more systematic manner to improve program quality and effectiveness.


Masters of Human Resource Management (MHRM) Outcomes Assessment Results

For Academic Year: 2015-16

Section I: Student Learning Assessment

Student Learning Assessment for (Program 1)

Intended Student Learning Outcomes for (Program 1) :

1. In the Spirit of Saint Francis in the Marketplace, students will be able to demonstrate in-depth knowledge of the ethical value choices and responsibilities of businesses and business leaders so that they will be able to engage in ethical decision making within a Franciscan context.

2. Students will demonstrate competencies in the following functional areas of HR; the foundation of HRM; staffing, recruitment, and retention; employment and labor law; compensation and benefits administration; performance management; and other content areas identified by the Society for Human Resource Management (SHRM).

3. Students will display skills in strategic analysis and integration of HR.

4. Students will have the ability to communicate with relevant stakeholders at a proficient (or advanced level) by developing (a) effective written communication skills, and (b) effective oral communication and business presentation skills.

5. Students will develop the ability to effectively work with a team of colleagues on a project.


1. Capstone HR Research Paper and Presentation in the MHRM 550 Strategic Human Resource Management course.

Using a research paper and presentation rubric, all students will either be “proficient” or demonstrate “mastery” of the competencies being assessed.


2. Comprehensive Human Resource Management Exam. Note: We are using the HR Review Capstone Exam. It is a three hour computerized exam consisting of 175 multiple-choice questions. The test is designed to measure HR knowledge and the ability to apply facts, concepts, theories, and analytical methods to “real life” HR professional situations. This exam was developed by HR Review, LLC, a leading provider of study materials to prepare HR Professionals for the Human Resource Certification Institute (HRCI) PHR/SPHR/GPHR certification exams.

A comprehensive test score of 70% is required to receive a “high pass” rating and certificate of accomplishment from the Dean. The MHRM Program Faculty have deemed a score of 50% to be “passing” during the first three years as we study assessment results and consider how to effectively calibrate our curriculum better with comprehensive HR test. The MHRM Program Faculty set the performance target that 75% of the MHRM students will earn a 50% or higher on the HR Review Exam. Eight graduates of the Loretto-based program completed the exam in the initial test of the instrument in 2014 and another 22 students in 2014-2015. Twenty-one students completed the HR Review Capstone Exam during the 2015-2016 Academic Year. The prior exam was the PHR professional certification test, but changes by the profession made this exam inaccessible to many of our MHRM graduates since they did not meet the two year professional experience threshold in an exempt HR staff position.


1. Annual MHRM Exiting Graduates’ Learning Self-Assesment and Satisfaction Survey, including perceptions of their learning outcomes and other programmatic-level outcomes. Revised ISLOs: ISLO 1. Students will demonstrate knowledge and competencies in the following functional areas of HR: the foundations of HRM; recruitment, selection, and retention; employment and labor law; compensation and benefits administration; performance management; and other content areas identified by the Society for Human Resource

Achieve at least a 4.0/5.0 or higher “mean” Likert scale satisfaction score on five of the key universal measures. A 4.0 indicates “agree” on a Likert scale and a 5.0 indicates “strongly” agree. Revision in Fall 2015: We assessed all six ISLOs in the Annual MHRM Exit Survey, effective Spring 2016. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to to be demonstrate knowledge and competencies in the following functional areas of HR: the foundations of HRM; recruitment, selection, and retention; employment and labor law; compensation and benefits administration; performance management; and other content


Management (SHRM). ISLO 2. Students will acquire effective professional skills, capabilities, and perspectives necessary for establishing and sustaining successful, productive careers in human resource management and related fields. ISLO 3. Students will demonstrate the ability to professionally communicate. ISLO 4. Students will display skills in strategic analysis and integration of HR. ISLO 5. Students will develop the ability to effectively work with a team of colleagues on a project.

areas identified by the Society for Human Resource Management (SHRM) to be "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their acquisition of effective professional skills, capabilities, and perspectives necessary for establishing and sustaining successful, productive careers in human resource management and related fields as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to to professionally communicate as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to more effectively display skills in strategic analysis and integration of HR as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to more effectively work with a team of colleagues on a project as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level.


2. Annual MHRM Graduate Career Placement Results Study. Achieve an annual placement rate of 90% or higher for MHRM graduates from the prior graduation year, 2015. That implies that they either found a relevant job in their chosen field or attended another graduate school within 6 months of graduation.


Met Not Met

1. All the graduate students achieved either “proficient” or “mastery” levels of performance on the Capstone HR Research Paper and Presentation in the MHRM 550 Strategic Human Resource Management course. Furthermore, several graduating students had their final thesis papers accepted for presentation and publication at the American Society for Competitiveness (ASC) Conference in Washington, D.C, in November 2015. This outcome validates that our graduates producing scholarship and research worthy of display and presentation at academic conferences. In Spring 2016 a new comprehensive research project assessment instrument was developed and used in the MHRM 550 Strategic Human Resource Management to access written communications skills, oral communications, analytical/critical thinking, and HR integration skills. Although not included in this final project rubric, other ISLOs were addressed through the class, including significant instruction time in applied managerial communications, collaborative/team work, group presentations, and peer evaluation.

X

2. Comprehensive Human Resource Management Exam. Note: We are using the HR Review Capstone Exam. It is a three hour computerized exam consisting of 175 multiple-choice questions. The test is designed to measure HR knowledge and ability to apply facts, concepts, theories, and analytical methods to “real life” HR professional situations. This exam was developed by HR Review, LLC, and a leading provider of study materials to prepare HR Professionals for the Human Resource Certification Institute (HRCI) PHR/SPHR/GPHR certification exams.

X


One student earned a 70% or higher on the HR Review Capstone Exam while 52% of the students achieved a 50% or higher on the exam (11/21). Since the performance target was 75% of the students, the target was not achieved.


Met Not Met

1. Annual MHRM Exiting Graduates’ Learning Self-Assessment and Satisfaction Survey, including perceptions of their learning outcomes and other programmatic-level outcomes. Select measures for three MHRM Class of 2014 include: As a result of the MHRM program, I have effective oral communication and business presentation skills: 4.67/5.0 in 2016 and *3.75/5.00 in 2015. As a result of the MHRM program, I have the ability to effectively work with a team of colleagues on a project: 4.50/5.0 in 2016 and 4.25/5.0 in 2015. MHRM professors displayed enthusiasm about their subject matter: 4.67/5.0 in 2016 and 4.75/5.0 in 2015. MHRM professors had the ability to clearly explain their subject matter: 4.5/5.0 in 2016 and 4.25 in 2015. MHRM professors devoted time to teaching real world applications of their subject matter: 4.67/5.00 in 2015 and 4.00/5.00 in 2015. I would recommend the MHRM program to other students: 4.5/5.0 in 2016 and 4.75/5.0 in 2015 The MHRM program staff was developed to providing excellent customer service: 4.67/5.0 in 2016 and 4.25/5.0 in 2015. Summary Assessment: All the performance targets listed above were achieved in 2016.

X


New Revision in Fall 2015: Student Measure their Progress on the Revised MHRM Intended Student Learning Outcomes. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to to be demonstrate knowledge and competencies in the following functional areas of HR: the foundations of HRM; recruitment, selection, and retention; employment and labor law; compensation and benefits administration; performance management; and other content areas identified by the Society for Human Resource Management (SHRM) to be "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. In-depth understanding of the foundations of Human Resource Management. Survey Result: 100% on both targets. In-depth understanding of staffing, which includes recruitment, selection, and retention. Survey Result: 100% on both targets In-depth understanding of employment law. Survey Result: 100% on both targets In-depth understanding of labor law. Survey Result: 83% on both targets In-depth understanding of compensation and benefits administration. Survey Result: 83% on the first target and only 33% on the second performance target. In-depth understanding of performance management. Survey Result: 100% on both targets In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their acquisition of effective professional skills, capabilities, and perspectives necessary for establishing and sustaining successful, productive careers in human resource management and related fields as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Result: 100% on the first target and 83% on the second one. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to to professionally communicate as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Result: 100% on both targets. In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to more effectively display skills in strategic analysis and integration of HR as "Average", "Good" or "Excellent"

X

X X X X X

X

X X


with at least 50% at the "Good" or "Excellent" level. Survey Result: 100% on both targets In Question 1 of the MHRM Graduate Exit Survey, 90% of students will rate their ability to more effectively work with a team of colleagues on a project as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Result: 100% on the first target and 83% on the second target In Question 1 of the MBA Graduate Exit Survey, 90% of students will rate their ability to identify and analyze ethical obligations and responsibilities of the HR professional and his/her organization as "Average", "Good" or "Excellent" with at least 50% at the "Good" or "Excellent" level. Survey Result: 100% on both targets.

X

X

X

2. Annual MHRM Graduate Career Placement Results Study. 93% placement rate for 2014 MHRM graduates with 94% of the graduates reporting (15/16). 100% placement rate for 2015 MHRM graduates with 71% of the graduates reporting (17/24). The average starting salary for MHRM graduates in 2015 was in the range of $40,000 - $49,999.

X


1. The MHRM Faculty have implemented the first year of the revamped MHRM 550 Strategic Human Resource Management using a revised comprehensive project assessment rubric that explicitly measures several of the MHRM program Intended Student Learning Outcomes (ISLOs). The rubric was developed by the course professor, Mr.Ken Tonkin,by modifying the IACBE Research Project


Evaluation Rubric to assess individual student performance. The final project and presentation represents 65% of the overall MHRM 550 course grade and is a central instrument to assess direct learning in the MHRM program. To this end, the following ISLOs were evaluated as part of this rubric:

• Written Communications Skills (part of SOB ISLO 3)

• Oral Communications Skills (part of SOB ISLO 3)

• Analytical/Critical-Thinking Skills (part of SOB ISLOs 4 and 6)

• Integration Skills (part of SOB ISLO 4)

Although not included in this final project rubric, other ISLOs were addressed through the class, including significant instruction time in applied managerial communications, collaborative/team work, group presentations, and peer evaluation. Based on first year assessment results, the performance target was met. However, it was not framed in a manner that is useful for assessment. Therefore, the new performance target moving forward is using a research paper and presentation rubric, 90%of the MHRM students will either be “Competent”” or “Exemplary” of the competencies being assessed. They include: Professional skills, capabilities, and perspectives necessary for the HR field; effective communication skills; strategic analysis and integration of HR; teamwork; and ethical obligations and responsibilities of the HR professional and his/her organization. The MHRM Program Faculty can expect to make additional refinements to this direct measure assessment tool in the future.

2. The MHRM Program Faculty implemented Year #3 of using the HR Review Capstone Exam and see if student results improve from the initial pilot for seven graduates two years ago. We need to calibrate the curriculum to the exam or see where the knowledge gaps are and develop an action plan to close the gap or better explain the variation. In 2013-2014, only one student out of eight completing the exam earned a 70% or higher on the exam in the first year, with a total of half of the graduating MHRM students achieving a passing score. In 2015, of the 22 MHRM students sitting for the exam, one achieved a high pass rating and a total of 10 achieved a passing grade. Two additional graduating MHRM students were just below the cut-off. In 2015-2016, 21 MHRM students completed the HR Review Capstone Exam, with one student scoring a 70% or higher and 52% of the MHRM students achieving the targeted passing score on it. Thus, we have yet to meet the performance target score of 75% of the MHRM students achieving a satisfactory score on the test. However, we have only begun to calibrate the curriculum to the NEW direct learning program assessment instrument. Furthermore, we need to incentize students to perform better on the test. Currently a target score on the test is not required to graduate and test results are not tied to course grades.


3. . The MHRM Student Exit survey was modified in Fall 2015 to explicitly measure the revised five Intended Student Learning Outcomes (ISLOS) and the results affirmed that students perceived that they are achieving these learning outcomes above the specificied targeted performance levels.

4. While the MHRM program faculty value career placement outcomes, this will be last year they are included in our indirect measure of student learning. They are appropriately measured under our operational assessment measures and not as indirect measures of learning.

5. We strive to improve our systematic use of program assessment results to improve the MHRM program’s quality and effectiveness. Section II: Operational Assessment (Note: Complete this section only if you received first-time accreditation or reaffirmation of accreditation after January 1, 2011.) Not applicable to Saint Francis University since we received full IACBE accreditation in 2007). However, we do report on our operational effectiveness measures being used for the undergraduate business program above, with many of them being equally applicable to the Graduate Business programs. Those proposed for the Graduate School of Business and Human Resource Management for 2013-2014 are listed below and will be reported on next year. We want to indicate voluntary adoption of this assessment system.


Section II: Operational Assessment (Note: Complete this section only if you received first-time accreditation or reaffirmation of accreditation after January 1, 2011.)

Operational Assessment for Saint Francis University Undergraduate Business Programs

Mission of the Saint Francis University Undergraduate Business Programs:

The Saint Francis University School of Business seeks to enable students and graduates to know more, do more, and be more. The School of Business will graduate students who have the knowledge, skills, love of learning, confidence, drive, and ethical and moral values needed to be spiritually fulfilled in life and successful in professional careers.

Intended Operational Outcomes:

1. The School of Business undergraduate program will be successful in placing graduates in appropriate entry-level positions or in graduate school on an annual basis.

2. Faculty members in the School of Business will be highly-qualified in their teaching disciplines and will be active in professional development activities.

3. The School of Business’ undergraduate faculty will be successful in providing high-quality instruction to its students.

4. The School of Business will be successful in providing effective academic advising to its undergraduate students.

5. Students in the School of Business undergraduate program will participate inrelevant internships, academic clubs, and other co-curricular activities on an annual basis.

6. The School of Business will carry out its mission in an efficient and financially feasible manner.


Assessment Measures for Intended Operational Outcomes: Performance Targets/Criteria for Operational Assessment Measures:

1. Saint Francis University Career Services Annual Placement Report

The School of Business will place 90% of its graduates in careers or graduates school within 1 year of graduation.

2. Saint Francis University Faculty Credential Files Faculty Annual Reports

1. At least 90% of the courses offered by the School of Business will be taught by either doctorally- or professionally-qualified instructors.

2. At least 50% of all of the school's full-time faculty will attend at least one discipline or instructional-development conference during the academic year.

3. At least 50% of the school's full-time faculty will either publish or present an academic article, poster, abstract or other work at an academic or trade conference.

3. IDEA Course Evaluations 1. At least 75% of the course offered by the School of Business will be above the national averages in the measures of excellence of teacher, excellence of course, and progress on relevant objectives.

4. Senior Exit Survey 1. At least 75% of senior students will indicate they are either satisfied or very satisfied with the academic advising of their business professors (Question #5)

2. At least 75% of senior students will indicate they are either satisfied or very satisfied with the instruction of their business professors (Question #5)

3. At least 25% of all School of Business Seniors will indicate that they participated in a credit-bearing or non-credit-bearing internship prior to graduation (Question #18)


5. Saint Francis University Academic Club Records

The active student enrollment in co-curricular activities such as SIFE, SEA, the Investment Club, or SBDC consulting work through SEED or EXCEL during the academic year will exceed 20% of the total undergraduate enrollment.

6. Delaware Study of Institutional Costs and Productivity FTE Students/FTE Faculty ratio will fall between 95% and 120% of national benchmark. The Direct Instructional Expense ratio will fall between 75% and 100% of the national benchmark.

Summary of Results from Implementing Operational Assessment Measures: Performance Target Was…

Met Not Met

1. Saint Francis University Career Services Annual Placement Report: According to the SFU Career Services report for graduates of the 2014-2015 academic year who responded to the survey, 100% of accounting undergraduates( 15 of 15) and 100% of all other business undergraduate majors (28 of 28) were either employeed full-time or were attending graduate school. For this same period, 91% (31 of 34) MBA graduate were employed full-time and 100% (17 of 17) MHRM graduates were employeed full-time.

X

2. Saint Francis University Faculty Credential Files Faculty Annual Reports During the Fall 2015 and Spring 2016 semesters, 100% of course offerings were taught by either doctorally or professionally-qualified instructors. As well, 72.7% (8 of 11) of the full-time faculty attended some form of instructional or professional development conference during the academic year and 54.5% 6 of 11) presented at a academic conference.

X


3. IDEA Course Evaluations For the fall 2015 semester, the percentage of School of Business courses offered by the School of Business that were above the national averages in the measures of excellence of teacher, excellence of course, and progress on relevant objectives. percentages were 79%,79%, and 87% respectively. For the spring 2016 semester, these percentages were 82%, 84%, and 88%.

X

4. Senior Exit Survey 93.48% of senior students indicated that they are either satisfied or very satisfied with the instruction of their business professors (Question #5) 80.43% of senior students indicated that they are either satisfied or very satisfied with the academic advising of their business professors (Question #5) 63.64% of all School of Business Seniors indicated that they participated in a credit-bearing or non-credit-bearing internship prior to graduation (Question #18)

X

5. Saint Francis University Academic Club Records According the academic club records, the number of participants in the SOB co-curricular activities were as follows: Society for Economic Analysis/Fed Challenge/Francis Fund 14, Enactus 21, Investment Club 9, SEED 7, EXCEL 1, CEO 5 for a total of 57 of the 213 students or 26.8%.

X

6. Delaware Study of Institutional Costs and Productivity X


According to the Delaware Study of Institutional Costs and Productivity, our School of Business FTE Students/FTE Faculty Ratio for 2013-2014 was 100.6% of the national benchmark (previous year: 107.2% of benchmark). While the Direct Instruction Expense per student was 77.2% of the national benchmark (previous year: 79.9% of benchmark)


Current Year For only the second time since we have been measuring our operational efficiency using the Delaware study, we met our benchmarks for both FTE Students/FTE Faculty Ratio and the Direct Instructional Expense ratio. Our benchmark is that FTE Students/FTE Faculty ratio will fall between 95% and 120% of national benchmark. The Direct Instructional Expense ratio will fall between 75% and 100% of the national benchmark. Not meeting these benchmarks was identified as an issue during the 2012-2013 assessment process and the School of Business submitted a proposal to add faculty in several area. At that time, the Provost’s Office approved a proposal for an additional accounting faculty member beginning with the 2013-2014 academic year. As a result, although we did not meet both benchmarks last year, we did meet one and saw improvement in both. The ratio of full-time student to full-time faculty decreased from 120.2% three years ago to 100.6% currently This is now within our target benchmark of 95% to 120%. The School of Business program direct instructional expense per student as a percentage of the national benchmark increased considerably from just 46.7% of the national business school average three years ago to 67.2% of the national average two years ago, 79.9% last year, and plateaued at 77.2% this. The additional full-time faculty member and a conscious effort by the School of Business to decrease average class size in the 2014-2015 academic year has paid dividends and placed in more inline with our peers.

Date post:	04-Jan-2017
Category:	Documents
Upload:	ngonhi
View:	213 times
Download:	0 times

View our IACBE Public Disclosure of Student Learning Form ...

Documents