A. John Bailerbaileraj@MiamiOH.edu

Providing better workforce preparation through experiential learning classes that promote collaboration and consulting

AbstractGenuine statistical collaboration and consulting experiences in the curriculum provide service learning opportunities where students can integrate community service into course content. They are also experiential learning opportunities where students engage with direct experience to increase knowledge and develop skills. Finally, these experience can provide a natural capstone to a degree program in statistics. In this talk, I will describe why these experiences are important, how this was implemented in the past, what are present and evolving practices and reflect on the value of this experience for students. I will provide a context for promoting these experiences based on recent revised guidelines for undergraduate statistics degrees and learning outcomes for masters degrees in statistics. I will illustrate the implementation of these ideas with data practicum and data visualization courses at Miami University.

Miami University? Oxford, Ohio?

University of Miami

http://www.contacttopuniversities.com/blog/mainmenu/University%20Articles/top-universities-for-physical-therapy/umEntrance.jpg

Miami University

Miami University

In my remarks, I will consider the general question of developing genuine statistical collaboration and consulting experiences in the curriculum

• Why is this important? Context for remarks

• Past patterns

• Present (& evolving) practice

• Reflection

Outline

ASA Workgroup on Master’s degrees in statistics(http://magazine.amstat.org/wp-content/uploads/2013an/masterworkgroup.pdf )Recommendations (based on survey of recent grads and employers):1. Solid foundation in statistical theory and methods.

2. Programming skills critical and should be infused throughout the graduate student experience.

3. Communication skills critical and should be developed and practiced throughout graduate programs.

Why is Important? Context of remarks

ASA Workgroup on Master’s degrees in statistics(continued)

4. Collaboration, teamwork, and leadership development should be part of graduate education.

5. Encounter non-routine, real problems throughout their graduate education.

6. Internships, co-ops or other significant immersive work experiences should be integrated into graduate education.

Context of remarks – graduate programs

ASA Undergraduate Guidelines Workgrouphttp://www.amstat.org/education/pdfs/guidelines2014-11-15.pdf

1. Increased importance of data sciencea. Access and manipulate datab. Perform algorithmic problem solvingc. Higher-level programming & database systems

2. Real applicationsa. Concepts and approaches for working with complex datab. Experience designing studies and analyzing non-text

book data

Context of remarks - undergraduate

ASA Undergraduate Guidelines Workgroup (ctd.)

3. More diverse models and approachesa. Predictive & explanatory modelsb. Model building & assessmentc. Apply theoretical foundation to sound analysis of data

4. Ability to communicate d. Communicate complex methods to managers and other

audiencese. Understand ethical standardsf. Opportunities to practice and refine skills during studies

Context of remarks - undergraduate

INGenIOuS (Investing in the Next Generation through Innovative and Outstanding Strategies) project (AMS,MAA,SIAM,ASA http://

www.maa.org/programs/faculty-and-departments/ingenious )1. Bridge gaps between business, industry, and

government (BIG) and academia.2. Improve students’ preparation for non-academic

careers - better preparation will increase the number of graduates who enter the workforce well equipped with skills and expertise in mathematics and statistics. Change is needed both in curricula and in some faculty members’ perceptions of BIG careers for their students.

3. Increase public awareness of the role of mathematics and statistics in both STEM and non-STEM careers.

Context of remarks – workforce prep. prof. societies

INGenIOuS (continued)4. Diversify incentives, rewards, and methods of

recognition in academia - A well-balanced mathematical sciences program offering a bachelor’s degree or above should include faculty with a variety of interests: discovery research (in pure and applied mathematics and statistics and mathematics education); work in applied, collaborative, and interdisciplinary areas; and teaching and preparation for careers both within and outside of academia.

5. Develop alternative curricular pathways.

6. Build and sustain professional communities.

Context of remarks

Miami University graduation requirements require:1. Experiential Learning http://www.units.miamioh.edu/oir/2020Plan/Experiential_Learning.pdfprocess of making meaning from direct experience in a real world or an “out of the traditional classroom” context2. Capstone coursehttp://www.miamioh.edu/liberal-ed/current-students/sr-cap.htmlcompleted near the end of baccalaureate studies, integrates liberal learning with specialized knowledge. Each Capstone emphasizes sharing of ideas, synthesis, and critical, informed reflection as significant precursors to action, and each includes student initiative in defining and investigating problems or projects.

Context of remarks – local concerns

Quick review of postings on ‘linkedin.com’ [ 17mar2015 ]

Data Mining / Predictive Analytics Consultant in São Paulo

Job description- Experience required:• Experience in implementations of Business Analytics solutions,

focused on Statistics and Data Mining;• Previous experience in at least one of these industries:

Financial Services, Insurance, Banking;• Previous experience with IBM Business Analytics software

(SPSS Statistics, SPSS Modeler, SPSS Analytical Decision Management);

• Previous experience with Data Mining and Statistics software from any other vendor.

What do employers want?

Skills required:• Excellent communication, interpersonal, and presentation

skills;• Academic background in Statistics and Data Mining;• Deep knowledge on implementation and use of Statistics and

Data Mining solutions.

Principal responsibilities:• As a Consultant within the Professional Services team your

primary responsibility will be to assist IBM’s high profile customers to achieve successful implementations of the SPSS/IBM Predictive Analytics Solutions.

What do employers want?

Initially, common features of early versions of data practicum classes included:• Problems described and motivated by the instructor using

artificially clean preprocessed data

• Labs were previously analyzed and a particular solution is likely

• Students alternated presenting by all students submitted reports of each analysis (often initial + final report)

• Stat instructor provided all feedback to oral/written reports

Past Patterns for data practicum classes

A company was thought to be polluting a local lake by discharging its manufacturing waste into the lake without pre-treatment. To investigate whether the lake was polluted, the EPA took five samples from the lake receiving the discharge (Lake #2) and five samples from a nearby unpolluted lake (Lake #1). Strontium measurements were recorded for each of the samples.

Data:Lake #1: 27.2 29.1 33.2 31.4 32.8 Lake #2: 37.4 35.0 41.2 40.6 36.2

Goal:Determine whether the strontium concentrations are different for the two lakes.

Requirements:Provide both graphical and numerical summaries as part of your analysis. All reports must be typed. Line printer plots are NOT acceptable.

Past Patterns for data practicum classes

Strengths of historical structure:1. Labs could be designed to span a breadth of

statistical methods2. Class was usually small (3-8) and students

had lots of chances to present.3. Opportunity to present ideas not formally

covered in other classes.4. Relatively homogeneous student population

(teams made up of students with similar majors)

Past Patterns for data practicum classes

Weaknesses of historical structure:1. Problems were already well formulated by a

statistician – no need to translate problem from a client

2. Data were preprocessed and relatively easy to mold into an analysis data set

3. Relatively homogeneous student population

Past Patterns for data practicum classes

Present (& evolving) practice

Current context – Moved undergraduate data practicum and graduate data practicum courses to client focused

Added data visualization class with multidisciplinary teams working on projects

Present (& evolving) practice

STA 475 Data Analysis Practicum (3) MPCThe use of statistical data analysis to solve a variety of projects. Emphasis on integrating a broad spectrum of statistical methodology, presentation of results both oral and written, use of statistical computing packages to analyze and display data, and an introduction to the statistical literature. A term project involving student teams combines elements of all of the above. CAS-QL. Prerequisite: STA 463/563 or 363; or ISA 291.

STA 660 Practicum in Data Analysis (3)Supervised practice in consulting and statistical data analysis including use of computer programs. Maximum of six hours may be applied toward a degree in mathematics or statistics. Offered credit/no-credit basis only.

STA 404/504 Advanced Data Visualization (3)Communicating clearly, efficiently, and in a visually compelling manner using data displays. Identifying appropriate displays based on various data characteristics/complexity, audiences, and goals. Using software to produce data displays. Integrating narratives and data displays. Critiquing visualizations based on design principles, statistical characteristics, and narrative quality. CAS-QL. Prerequisite: at least one of the following: STA 261, 301, 368, 671; IMS 261; ISA 205; or by permission of instructor. Cross-listed with IMS/JRN.

Present (& evolving) practice

Desire: 1. Direct engagement in wrestling with client-defined

tasks

2. writing outcomes

3. group work

4. service learning

Present (& evolving) practice

Challenges and implementation

1. Getting Clients?

2. Projects

3. Reflection

Getting Clients?

Need to actively recruit clients and screen projectsTargeted email …

SUBJ: An invitation to propose projects for data analysis capstone / practicum class

Greetings, Have you or your office collected data that you haven't had the chance to analyze? Are you planning for future studies and would like some assistance determining how many observations you might need? Do you like working with motivated students? If you can answer "yes" to any/all of these questions, then I invite you to put my students to work …

Getting Clients? (continued)

If you have a project(s) where statistical assistance might be valued, then let me know. Please send me a short description of the project including:(on the email subject, please use the convention - SUBJECT: STA 475 project: your name - project title) 1. Short descriptive project title 2. Goal of the analysis (e.g. design phase - project planning; data analysis, etc.) 3. Data to be analyzed (e.g. Excel data sheets; still to be collected; ...) 4. Type of statistical analysis anticipated (e.g. logistic regression, anova models, etc.) 5. Timeline for analysis (when are results needed)

Getting Clients? Data Visualization

Happy New Year. I hope that you are doing well. We are teaching the data visualization class (IMS/STA 404-504) for a second time this spring. This course will be populated by undergraduates and graduate students representing a diverse set of backgrounds including design, finance, psychology and statistics. Our course will focus on the construction of well designed data displays that tell accessible stories from data. A major component of this class is a project that will be conducted for an external client, and we thought you might be interested in participating as a client.

Do you or your office have data that would benefit from better analysis and visual display? Do you have a complicated story involving numerical summaries in which visualization might lead to insight? Do you have data that you haven’t fully investigated but you believe might contain the nugget of an interesting story? Do you have reports that are overly dense with tables and basic graphics that would benefit from redesign? If you are interested in help addressing these issues, we invite you to submit a project idea for consideration.

Getting Clients? Data Visualization

As an external client, you would meet with the class early in the semester (mid-February) to outline the project and desired outcomes, in the middle of the semester (late-March) to review interim results and prototypes of visual displays and at the end of the semester (early-May) to receive and review the final project. Ideally, a client would be available to meet on campus, but we could also arrange WebEx/Skype as well. We will be supervising the students as they work on these projects.

If you have a project that you think might be appropriate, please let us know (baileraj@MiamiOH.edu; kuhnar@MiamiOH.edu ).

Getting Clients? Data Visualization

Send us a short description of the project, including

1. A short title;2. Goal of the analysis (e.g. dashboard displaying important data; website with interactive

visualization; a story for possible print/web publication);a. If possible, provide at least one or two specific questions to be answered by the

analysis3. Data to be analyzed? Source of data? Format? (e.g. Excel spreadsheets, CSV files);4. Publications or report that have previously presented the data5. Type of statistical analysis anticipated

Getting Clients? (continued)

Observations:Do a good job once and clients will return in future.

Current repeat clients in my data practicum class:GerontologistsExercise Physiologist

Current repeat clients in data visualization class:Local paper – Cincinnati Business CourierResearch center – Scripps Gerontology Center

Projects

I am the first client (probably the worst they will have)

Project: compare dissolved oxygen-depth relationship between two lakes

Very general guidance on first draft report

Extensive commenting on first draft to be addressed with revision (at least one revision)

Projects (continued)

Ideas to convey early …1. Revision and critical reading of reports key skill and learning outcome2. Better graphical displays lead to easier writing and communicating with clients3. Reporting effect estimates often richer than exclusively reporting the results

of hypothesis testing { indicates what students are taking away from our classes }

4. Writing a structured report is a skill

Projects (continued)

Later projects – teams; charter-traditional schools; CELTUA; OMA; development)• Project life course• Client meeting and discussion with class• Team work• Interim reporting of analyses• Draft report produced (class wiki for other teams to comment)• Client presentation of final analysis• Final report and presentation sent to client

Projects – Example OPTAB

Task: conduct an occupancy study of the various parking areas in town so that the Oxford Parking & Transportation Advisory Board (OPTAB) could make data-based decisions regarding meter rates, hours, fines, etc.

Client: board which represents nearly every constituency is town: City Council, Miami University, Chamber of Commerce, Oxford Landlords, Mile Square Residents, Year Round Residents, ASG, Talawanda Schools and McCullough-Hyde. Ex- officio members include Chief of Police and the City Manager

Projects – Example OPTAB

Issues: When a block face, i.e., the collection of parking spaces on a street for a city block, exceeds 85% occupancy it appears full.

If the remaining 15% of block face spaces are vacant, then this would lead to a loss of earnings (if these are metered spaces).

City parking garage has low occupancy.

Projects – Example OPTAB

Conduct an analysis of a sample of the 790 metered spots in Oxford as well as the parking garage.

Goal: investigate different rates based on location to spread occupancy from “hot spots” to outer locations as well as into the unoccupied city garage. In addition, ‘meter feeding’ could be removing spaces from circulation for extended periods of time.

Question: extending meter hours past the current 6 p.m. deadline which could force many vehicles to park away from the high and main intersection as well as produce additional income for the City of Oxford.

Projects – Example OPTAB

Solution:Developed a data collection plan that was implemented in March of 2010. Four days (Monday, Thursday, Friday and Saturday) were sampled hourly (11 a.m.-3 p.m.; 4 p.m.-8 p.m.) - restricted attentions to 27 block faces and the parking garage for a total of 377 parking spaces.

Logistics of the data collection:order that spaces were checked; data collection sheets; decisions about what to record such as license numbers) appropriate documentation and emergency numbers.

Projects – Example OPTAB

Solution:Developed a data collection plan that was implemented in March of 2010. Four days (Monday, Thursday, Friday and Saturday) were sampled hourly (11 a.m.-3 p.m.; 4 p.m.-8 p.m.) - restricted attentions to 27 block faces and the parking garage for a total of 377 parking spaces.

Logistics of the data collection:order that spaces were checked; data collection sheets; decisions about what to record such as license numbers) appropriate documentation and emergency numbers.

Projects – Example OPTAB

For each metered spot, occupancy was recorded and whether the meter was in violation. Summaries of the collected data included:

Occupancy/Availability rate per time, day and block on street parkingOccupancy /Availability rate per time, day in garageViolation rate per time, day and block on street parkingViolation rate per time, day in garageMeter feeding: less than 2hrs, occupied over 3hrs including location

Results reported to the OPTAB clients included: Satellite Map with highlighted linesHeat map

Projects – Example OPTAB

Example of Heat Map of occupancy for one sampling time

Projects – Example OPTAB

Occupancy of block face over time on different days

Projects – Example OPTAB

Impact Students presented the results of this data collection and analysis effort to the OPTAB. The board was very impressed with the depth, quality and insight provided by this project. The board mentioned that these data would be relevant for setting parking meter rates that might include differential rates for underutilized spaces. Finally, the OPTAB members commented that this level of work may have exceeded the value of a previous report that cost over $20000 to conduct. At this point, the students were left speechless.

Projects – Example OPTAB

Follow-upExperience of working with a local government organization and contributing to information needed to support decision making

Students had to coordinate an extensive data collection effort, process the data into an analysis data set, construct displays and summaries that gave insight and develop a presentation and report that captured all of this work.

Comment about the cost of previous parking studies led to interesting class exercise of developing a cost estimate of how much billable work was reflected in the analysis, and what they would charge if they did this as a consulting company

Projects – Example Data Visualization

Advanced Data Visualization class (team taught by graphic designer and statistician)Final Client Project (25% of grade). Students create a visualization integrated with a brief narrative using data provided by an outside client to solve a problem or answer a question (“Describe clients receiving emergency services”, “Display budget of city”, etc.). Students will work collaboratively with faculty and clients in the development of the solution. Client and faculty will collaborate on final grade. Students will conduct peer reviews, which will factor into final grade as well.

http://dataviz.miamioh.edu/UnitedWay/

http://dataviz.miamioh.edu/CBC-Cincy-Budget/

Projects – Example Data Visualization – peer review404/504 Peer Feedback for Final Client ProjectPeer Name (required) Ratings: (Terrible) 1 … 2 … 3 … 4 (Great)• Attendance at meetings held outside of class by the team:• Attendance at meetings held outside of class by the team:• Amount of contribution outside of team meetings to the team's sprint tasks:• Quality of contribution outside of team meetings to the team's sprint tasks:• Effectiveness in working collaboratively (e.g. sharing decision-making with

others, soliciting other's ideas, encouraging, energizing):

{ Client assessment given in Appendix of extra slides }

Projects – Example Data Visualization – peer review404/504 Peer Feedback for Final Client ProjectPeer Name (required) Open response• Relative to other team members, how would you describe this person's

contribution?• What are this person's major strengths? What have they brought to the table

for your team?• How could this person improve his/her effectiveness as a member of your

team?

1-10 (10 being highest), how likely would you be to recommend this person to another team? [1=‘Do not work with them’ … 10=‘Work with them!’

Reflection

Assessments:

Learning outcomes (BS STA degree)1. Students shall be able to analyze and interpret data critically using statistical

models and programming skills { also part of practicum }2. Students shall demonstrate understanding of the mathematical basis and

theoretical foundations of statistics3. Students shall be able to effectively communicate, both orally and in written

form, results of statistical analyses to both the expert and layperson { data practicum class critical data for evaluating this LO }

Reflection (continued)

Reflection - STA 475 Data Practicum Portfolio and Self evaluation“Each Capstone emphasizes sharing of ideas, synthesis, and critical, informed reflection as significant precursors to action, and each includes student initiative in defining and investigating problems or projects” (http://www.units.muohio.edu/led/Capstone). Your final exam is a paper reflecting on the following questions derived from the reflections associated with service learning courses. What?What projects did you work on this semester?What was your role on project teams? Note that this may change on different projects.What did you observe?What did you like/dislike about working on these projects?What missing knowledge or skills would have made you a more effective contributor to the work on these teams?

Reflection (continued)

So What?What stuck out about the experience in this capstone?What was the best/worst thing that happened?What have you learned about yourself?How does this experience compare to others you’ve had? Now What?What have you learned about working as a statistical collaborator/consultant?How did this experience challenge you?Do you believe that this course prepared you for future collaborative work? If so, how? If not, why? SummaryWhat grade would you assign to your efforts this semester? Why?Describe the work or contribution of particular individuals to project teams that you thought was noteworthy.

Reflection (continued)

Reported from students … during interviews with potential employers

Employers expect graduate will have the stat and math background but look for the experience with client (often messy) problems, where they worked in teams and had to produce written and oral reports for the clients.

Experiences such as described above provide this and I encourage you to develop (as academics) and to support (as potential clients)

Thank you

Rubrics - presentations

Category Does Not Meet Expectations Meets Expectations Exceeds Expectations Written report Description of project/questions is inaccurate or insufficient. Results of

statistical analyses poorly described. Conclusions inadequately address questions. Assumptions used in analysis are unstated or invalid. Organization is awkward. Literature citations are missing or inappropriate. Graphical and tabular displays are missing or uninformative. Errors in language mechanics (grammar, spelling) are frequent or serious.

Project is introduced accurately. Principal questions, methods and results of analyses (including assumptions), and conclusions addressing client objectives are all clearly stated. Report is organized sensibly and includes appropriate graphical and tabular displays and literature citations. Occasional errors in language mechanics might be present but aren’t distracting.

Meets expectations and… Writing is fluid and succinct. Description of methods, analyses, results and recommendations are presented authoritatively and convincingly. Validity of analysis has been evaluated and confirmed. Graphical and tabular displays are well-chosen, carefully formatted and correctly referenced. Errors in language mechanics are rare and minor.

Communicating results (applies to both written and oral presentations)

Conclusions are lacking or unclear. Descriptions of statistical procedures are overly technical or missing. Analysis is inadequately connected to the project’s background and needs. Graphical and tabular displays are difficult to read and fail to provide insight into the problem or its solution. Statement of statistical techniques is omitted or described in vague terms. No connection is given to statistical literature. Terminology is used incorrectly.

Conclusions are stated clearly and are connected to the project's original questions. Descriptions of statistical procedures are described at an appropriate technical level. Analyses connect directly to the project’s objectives. Appropriate citations to statistical literature are given. Terminology is used correctly. Graphical and tabular displays emphasize insight regarding conclusions.

Meets expectations and… Analyses and descriptions enhance and extend the understanding of the project and its goals. Provides comprehensive, elegant, and well-sourced presentation. Terminology is used precisely. Relevant diagnostics are discussed and interpreted correctly.

Oral present-ation Order of slides is illogical or unintuitive. Many slides are too dense or incomprehensible. Speaker is difficult to understand, seems unprepared, or answers questions awkwardly. Font sizes and colors make slides difficult to read. Talk is overly detailed and too long, or is underdeveloped and too short. Speaker struggles to respond to questions.

Slides are logically ordered and mostly uncluttered. Each slide either provides a summary or focuses on a single point of discussion. Font sizes and colors are appropriate for presentation media and venue. Presentation is well paced and of appropriate length. Speaker is understandable, well-prepared, and answers questions comfortably. Speaker discusses each slide, but does not read from slides.

Meets expectations and… Slides provide a strong introduction, smooth and logical flow throughout, and clear statement of conclusions. Presentation is engaging, polished, and notably enhanced by careful choice of displays and formatting. Speaker is able to respond to questions aptly and concisely.

Rubric for assessing MS Statistics, Learning Outcome 3: Students shall be able to effectively and independently communicate, both orally and in written form, the results of statistical analyses. [Applied to the Master’s oral exam and accompanying written report of the Final Project for the MS in Statistics.]

Rubrics – presentations to clients

STA/IMS 404 Client Project - Client EvaluationEvaluator Name:Client:Students:

{ all questions rated from “low” to “high” }• How would you rate the project’s underlying concept (interesting story, display answers an interesting question, etc.)?• How would you rate the project’s technical level?• How would you rate the analysis done to execute the project (is the display accurate and appropriate, is there evidence of

deeper analytical thinking and insight)?• How would you rate the aesthetic qualities of the project (is it aesthetically pleasing, are colors and typography

appropriate and consistent, etc.)?• How likely would you be to actually implement some of the ideas presented by the students?• How does the project align with what your expectations were?• Based on this project, how likely would you be to recommend these students to a colleague faced with a similar

problem?{ open ended }

Final thoughts?