Date post: | 21-Jan-2016 |
Category: |
Documents |
Upload: | jean-lloyd |
View: | 213 times |
Download: | 0 times |
A Comprehensive Assessment Approach for the School of Advanced Studies
Rob Olding, Ph.D. Associate Dean for AssessmentUniversity of Phoenix: School of Advanced Studies
Learning Assessment for Doctoral Level Education is …?• Relatively new in current conceptualization• Relatively abstract (compared to Associate and
Bachelor levels)• Higher thresholds for expectations of existing
skills• Research and writing intensive
A Multiple Method Approach Assessment Strategy Somewhat in the form of an “Instrumental
Case Study” approach consisting of “data vectors” from various perspectives: Rubric based assessment based on IRMA mapping of
curriculum External evaluation sources (e.g., accreditor reports,
Council on Graduate Schools, external consultants). Appreciative Inquiry approach to assessment
Triangulation of Assessment Data Sources (basic model)
SAS Practitioner Doctorate Learning Assessment Data
Sources
Rubric based measurement: Student Learning Outcomes
External 3rd Party Data (Reports)
Appreciative Inquiry (AI) Based Assessment
Deliberative Analysis
Rubric based measurement: Student Learning Outcomes
Critical to the assessment process is the development of analytic rubrics to measure performance on signature assignments as described.
As documented in multiple sources, analytic rubrics have been shown as effective feedback even at the advanced education level (Groggins-Selke, 2013)
Analytic Rubrics focused upon:
Signature assignments in content and research courses
Key Progression assignments such as dissertation concept paper and proposal
Doctoral Dissertation assessment via multiple reviewers as “capstone” measure of doctoral success
External 3rd Party Data (Reports)
Leveraging the data from these different sources to form a clear analytical picture of how our learning outcomes and programs are perceived from external views.
External Consultant Reports as engaged by the University of Phoenix School of Advanced Studies – (e.g., Council of Graduate Schools)
Survey Data (e.g., SEOCS and Alumni Surveys)
Accreditor Reports (e.g., Higher Learning Commission)
Appreciative Inquiry approach to Learning Assessment This approach is argued to be
atypical in assessment of learning, but very appropriate for the Doctoral level which compared to lower level degrees can be diverse, abstract and dynamic in outcomes.
As a method is known to be “transformational” and “engaging” which is important for both faculty and students
Positive in focus: “What were the most important learning outcomes you found in this course”?
Tied directly to a process of envisioning what we do best, what our dream outcomes would be, and how we might move to obtain that level!
Appreciative Inquiry Steps: 4-D Modeladapted from Watkins and Mohr (2001)
Discovery“Best of what is?”
Appreciating
Dream“What might be?”
Envisioning Results
Design“What should be the
ideal?”
Destiny“How to empower, learn, improve?”
Sustaining Change
The three approaches to assessment data provide a diverse set of perspectives
These perspectives can be left unreconciled, or…
Integrated to provide a new level of perspective that informs far beyond any on its own.
TRIANGULATION
Integration of Multiple Data Sources using diverse methods
Includes diverse perspectives in the discussion
Provides for a basis of meaningful engagement for administration, faculty and students
Avoids an over reliance on a single paradigm approach that limits perceptions
Advantages:
The Triangulation process requires a means of bringing together diverse data sources
An ideal approach would extend the process to multiple “stakeholders” that would include students, faculty and administration
Such a method would work to ensure “fair consideration” of each data source in relation to optimum learning outcomes
Means of Integration:
The Deliberation ModelAn inclusive approach that involves all stakeholders and represents each of the data sources in the process
Deliberation: A means of integrating diverse perspectives
Deliberation is neither a casual discussion or a formal or informal debate. The discussion should focus first on each source individually and then in combination and from a “best solution” view
Maximum engagement – Can be used in face to face, online synchronous and asynchronous formats
Allows for carefully weighing diverse views and bringing light upon points not previously considered and explored
Allows for exploration of facts in relationship to strategic positions as represented by each area data source
External Reports (3rd party)
Rubric based Assessment
Appreciative Inquiry Results
Interpretation and
Integration
Each perspective is sequentially discussed in terms of meaning and both positive and negative implications. Then they are compared and discussed in relationship to integration with each other. Commonalities are of particular note in TRIANGULATING meaning.
The strategy is to: Maximize engagement of stakeholders Optimize the data sources in ensuring that
diverse methods and perspectives are included in consideration
Develop an integrated set of insights that can inform strategy and contribute meaningfully to continuous improvement
Move assessment and evaluation to a new level for Doctoral Education
Insights and Outcomes:
Thank You!
Questions????
REFERENCES
Borkowski, Nancy, A. (2006). Changing our thinking about assessment at the doctoral level. In Maki,
Peggy L. & Borkowski, Nancy A. (Eds.), The assessment of doctoral education:
Emerging criteria and new models for improving outcomes (pp. 11-51). Stylus Publishing LLC,
Sterling, Virginia.
Bushe, Gervase R. & Kassam, Aniq F. (2005). When is appreciative inquiry transformative?: A
meta- case analysis. The Journal of Applied Behavioral Science, Vol. 41, No. 2, pp. 161-181.
DOI: 10.1177/0021886304270337.
Groggins Selke, Mary J. (2013). Rubric assessment goes to college: Objective, comprehensive
evaluation of student work. Rowman & Littlefield Education, Lanham, Maryland.