1

Assessing the Academic Climate

Bruce Szelest & Jeff Gerkenbszelest@uamail.albany.edujgerken@uamail.albany.edu

AIRPO Annual ConferenceStony Brook, New York June 19-21, 2006

2

Personal Traits

Pre-CollegeCharacteristics

Parents’ Education

Aptitude TestScores

High SchoolAchievement

CollegePreparation

Involvement

Receptivity to ServicesEducational Aspirations

Age, Ethnicity,& Gender

College Experiences

Acad Integration

Social Integration

Acad ConscientiousnessClassroom Experiences

Faculty ContactField of Study

Peer RelationsEmployment

Extra-Curr. ActivitiesResidential Experience

Educational Outcomes

Arts & Letters

Disciplinary Study

Scientific Method

Academic Achievement

Persistence/Graduation

Alumni Outcomes

Graduate DegreesEarned

OccupationalStatus

Income Level

Satisfaction withJob and Career

Leadership& Service

Awards& Recognition

Personal

Interpersonal Skills

Openness & Tolerance

FunctioningIndependently

Parental Influence

Inst’l Integration

Inst’l CommitmentAdvisement

Financial AidAffinity of Values

AlumniGiving

Academic

Acad Motivation: Intrinsic, Extrinsic

Self-regulation

The Albany Outcomes Assessment Model

3

The Albany Longitudinal Survey Program

Year 1 Year 2 Year 3Year 4+

Entering Student Senior ExperienceSurvey Survey

Freshmen StudentExperience Survey

4

Entering Student Survey

•Focus on student pre-college characteristics to identify students at risk prior to first semester of study

•Concentrate institutional resources on students in-need of attention - individual student profiles

•Communicate general findings back to new freshmen to raise awareness of successful student profiles

•One leg of the Advisement Services Center’s three-legged assessment program

•http://web.albany.edu/ir/samplesurveys/ess.html

5

Parental InfluenceComposite of mother's and father's educationMy parents encourage me to do the best in whatever I doI can go to my parents when I have a problem

Educational Aspirations (scale: importance)To gain a broad, well-rounded education (in the sciences, humanities, and arts)To learn to think creatively and analyticallyTo learn more about myself, my values, and my life's goalsTo learn more about other cultures and groups of people.

Involvement (scale: degree of chance)Participate in intercollegiate or intramural sportsBelong to a fraternity or sororityJoin a student club (academic, political, or cultural)Participate in student government, student newspaper, and/or campus radio station

Receptivity to Services (scale: degree of chance)Visit professors during their office hoursAttend a study group from one of your classesContact my academic advisor more than once per semesterVisit the Career Development Center

College Preparation (scale: yes or no)Pre-calculus or calculsForeign languageScienceAdvanced placement (AP) courses

Pre-college Scales

6

Pre-college Scales – cont’dAcademic Motivation - Intrinsic (scale: not true to true)

I enjoy the classes that challenge my mind.I will feel successful if I have the highest test scores.Thinking about my future motivates me to work hard in school.Solving difficult problems in my classes gives me a sense of satisfaction.I feel successful when I learn something interesting.Learning new things gives me a sense of accomplishment.Schoolwork is interesting because you see yourself making progress.I read books not assigned in classes.

Academic Motivation - Extrinsic (scale: not true to true)I study hard because I want to prove myself as capable as anyone else in class.I worry about not doing as well academically as others in college.I don't feel I can gain much in college other than obtaining a degree.I feel that as long as my teachers are satisfied with my academic performance, that is

enough.I don't really care how well I will do in college, as long as I can pass.Getting a high paying job is the main reason I study hard in college.I feel ashamed if I don't do as well as other students do in college.I worry about failing exams and getting into academic trouble.

Self-Regulation (scale: not true to true)When I get something wrong in school, I always stop and try to find out what went wrong

rather than simply asking my instructor the correct answer.I take notes while reading textbooks.When I run into difficulties in schoolwork, I feel if I put in extra effort, I will eventually figure them out.When I am reading my textbooks, I stop and check whether I really understand what I have

just read. …cont’d

7

Student Experience Survey (frosh & seniors)

•Examine changes in students’ psychological make-up after the first college year

•Gauge institutional engagement and relationships with educational outcomes

•First pass at gauging centralized student advisement

•Communicate general findings back to students and faculty to support the development of a campus culture of assessment

•http://web.albany.edu/ir/samplesurveys/froshses.html

8

College Experience Scales & Items

Academic Conscientiousness (scale: various 5 pt scales)How frequently have you worked harder than you thought you could to meet an

instructor's expectations?Developed your study skills.Utilized your study skills.I do very little studying on weekends.I put a good deal of effort into being well prepared for examinations. My primary goal at UAlbany is to take advantage of academic opportunities.Doing homework and studying outside of class.

Classroom Experiences (scale: never to almost always)Been satisfied with your academic experiences?Had out-of-class assignments that were good learning experiences?Worked with other students on class assignments?Received feedback (written or oral) from instructors on the quality of your work?Had to critically appraise the value of information, arguments, or methods in your classes

or class assignments?

Faculty Contact ItemsHow frequently have you had discussions, meetings, or conversations with instructors

outside of class?

Visit professors during their office hours?Seek out a faculty mentor?

Academic Integration:

9

Social Integration:

College Experience Scales & Items– cont’d

Peer Relations (scale: disagreement to agreement)I have developed strong friendships with other students. My interpersonal relationships with other students have had a positive

influence on my personal growth.The student friendships I have developed have been personally satisfying.It has been difficult for me to make friends with other students.I know several UAlbany students who would be willing to help me if I had a

personal problem.My interpersonal relationships with other students have had a positive

influence on my intellectual growth.I am confident in my ability to make friendships.

Extra-curricular activities (scale: yes or no)Participate in intercollegiate or intramural sports?.Belong to a fraternity or sorority?

EmploymentWorking on-campus (volunteer or paid).Working off-campus (volunteer or paid).

10

Institutional Commitment (scale: yes or no)Visit the Writing Center?Visit the Career Development Center?Contact your academic advisor more than once per semester?

Advisement Items (scale: disagreement to agreement)Advisor was knowledgeable about course requirements for the major(s). Advisor was knowledgeable about course requirements for the minor(s). Advisor was helpful in assisting with scheduling/ registration procedures. Advisor showed genuine interest in my academic progress.Advisor went over my degree audit with me to inform me of course requirements I

have not yet met.Advisor was knowledgeable about academic policies (e.g., graduation

requirements, GPA requirements, General Education guidelines, residence requirements, etc.).

Advisor was available when I needed him/her.Advisor showed concern for my personal growth and development.Advisor helped me to identify career areas (or graduate school opportunities)

which fit my skills, abilities, and interestsAdvisor encouraged me to talk about myself and my college experience.Advisor was easy to talk with. Advisor promptly returned my phone calls, e-mails, and/or messages.

Institutional Integration:

College Experience Scales & Items– cont’d

11

Arts & LettersWriting effectively.Developing intellectual curiosity.Developing the ability to formulate creative ideas and/or solutions.Appreciating artistic and creative expression.Placing current problems in historical perspective.

Scientific MethodEvaluating ideas, materials, and methods critically.Developing problem solving skills.Thinking analytically and logically.Understanding scientific findings.Understanding mathematical concepts.

Disciplinary StudyGaining factual knowledge.Learning how to learn.Synthesizing a body of information.Understanding a particular discipline's research methods.Understanding a particular discipline's various schools of thought.Understanding the inter-relatedness of different fields of study.Using information and/or computer technology in your academic discipline

Educational Outcomes

Academic (scales: none to very large college contribution)

12

Educational Outcomes – cont’d

Social (scales: none to very large college contribution)

Interpersonal SkillsSpeaking effectively. Developing interpersonal and social skills.Functioning effectively as a member of a team.Preparing for active participation in a democratic society.Coping with conflict.Adapting to different social situations.

Openness & ToleranceUnderstanding cultural differences.Gaining exposure to a variety of new intellectual areas.Developing an openness to new ideas.Coping with moral and ethical issues.Developing a better understanding of myself (e.g., interests, talents, values,

limitations).Understanding cultural differences.Relating well to people of different races

Functioning IndependentlyFunctioning independently (self-reliance).Exercising personal responsibility.Developing a better understanding of myself.

13

Personal Traits

Pre-CollegeCharacteristics

Parents’ Education

Aptitude TestScores

High SchoolAchievement

CollegePreparation

Involvement

Receptivity to Services

Educational

Aspirations

Age, Ethnicity,& Gender

College Experiences

Acad Integration

Social Integration

Acad ConscientiousnessClassroom Experiences

Faculty ContactField of Study

Peer RelationsEmployment

Extra-Curr. ActivitiesResidential Experience

Educational Outcomes

Arts & Letters

Disciplinary Study

Scientific Method

Academic Achievement

Persistence/Graduation

Alumni Outcomes

Graduate DegreesEarned

OccupationalStatus

Income Level

Satisfaction withJob and Career

Leadership& Service

Awards& Recognition

Personal

Interpersonal Skills

Openness & Tolerance

FunctioningIndependently

Parental Influence

Inst’l Integration

Inst’l CommitmentAdvisement

Financial AidAffinity of Values

AlumniGiving

Academic

Acad Motivation: Intrinsic, Extrinsic

Self-regulation

Entering Student SurveyID at-risk studentsGeneral profiles of successful students

Freshmen & Senior SurveysExpansion of the Albany ModelGauge institutional engagementEvaluate advisement (centralized & in the major)Build culture of assessment (generally & in acad units)

NSSE

14

The IR Survey Program in Action

Communicating Results and Next Steps

15

16

Chances Fall 2004 Frosh will transfer to another college before graduatingBY

Enrolled one year later in Fall 2005

35 315 350

10.0% 90.0% 100.0%

102 655 757

13.5% 86.5% 100.0%

65 242 307

21.2% 78.8% 100.0%

17 78 95

17.9% 82.1% 100.0%

17 46 63

27.0% 73.0% 100.0%

236 1336 1572

15.0% 85.0% 100.0%

Count

% within Will transfer to

another college before

graduating

Count

% within Will transfer to

another college before

graduating

Count

% within Will transfer to

another college before

graduating

Count

% within Will transfer to

another college before

graduating

Count

% within Will transfer to

another college before

graduating

Count

% within Will transfer to

another college before

graduating

1 no chance

2 slight chance

3 moderate chance

4 good chance

5 almost certain

Chances will

transfer to another

college before

graduating

Total

No Yes

Enrolled One-year

Later

Total

17

18

19

20

21

22

Strategies for Affecting the Course Experiences Dimension

Earth & Atmospheric Sciences - “treat them right”

1. Interaction space - Weather center; Map room• Develop sense of community with other

students• Interact with graduate students• Faculty in and out all day

2. Take undergraduates to conferences3. Undergraduate leadership conference4. Pre-graduation dinner and roast for graduates,

families, and professors5. Faculty available regardless of office hours,

everyday

6. Make students feel part of something more intimate than the greater University

23

Strategies for Affecting the Course Experiences Dimension

Public Policy – Attention to the curriculum

1. Cluster of faculty review the intentionality of the curriculum

2. Senior seminar (capstone course)• FT faculty involvement• A lot of one-on-one contact with faculty• Research paper• Writing intensive, and oral discourse

components3. Moved away from disproportionate instruction by

doctoral students• Careful mentoring of doctoral student

instructors• 1st teach with a full-time faculty member• Review performance

4. Make the final experiences for students a positive one

24

Strategies for Affecting the Course Experiences Dimension

English – Personalizing a large major

1. Smaller classes at the Upper Division2. Juniors and seniors get the most experienced

instructors in their areas of interest3. Undergraduate research conference each Spring

• Interaction with FT faculty• Guest speaker

4. Active honors program5. Very active chapter in National Honor Society

25

Strategies for Affecting the Course Experiences Dimension

Business Administration – Focus on the pragmatic

1. Meaningful project experiences - a lot of group work

2. Focus on presenting student work• Opportunities for collaboration• Consulting reports• Research projects

3. Active use of SIRF (by dean and chairs)• Work with faculty who can benefit from further

development• Master teachers program

4. Capstone course – wrap it all up for students

26

27

28

29

Successful Approaches to Advisement in the Major

Earth & Atmospheric Sciences – Advise beyond the norm

1. Make effort to sit down with every student (even intended majors)

2. High involvement of FT faculty3. Divide up advising among professors4. Training sessions (informal) for professors5. FT faculty, regardless of office hours, available

everyday

30

Successful Approaches to Advisement in the Major

Biological Sciences – Lot’sa Majors, personal attention

1. Faculty advisor for every student (25-30 students/FT faculty)

2. Junior faculty have fewer advisees than senior faculty

3. Coordinator for undergraduate advisement• Gate keeper, assign students to advisors,

approve advisor changes• Handle course substitutions, waivers, dropping

issues, etc.4. Transfer advisement team (faculty and staff)

• Trained to interpret degree audit, PS functionality

• Registration and scheduling help• Bring issues of import to students to faculty

attention5. Availability of written information and policies

• FAQs in central locations• Advisement info on department web site• PS Data Warehouse to get student e-mail

addresses

31

Successful Approaches to Advisement in the Major

History – Hybrid approach

Historically:• Very decentralized approach, individual faculty kept

advisee files

Now:1. Centralized file locations

• Keep notes on student intentions, advisor comments, etc.

• Print degree audits on-site2. Students make appointments with FT faculty in

centralized location 3. FT faculty take turns handling walk-in advisee

needs (advise & forms available)4. Recommend each major select an individual FT

faculty advisor (promote mentoring relationships)5. Advisement on the www, FAQ6. Survey of majors, get info on student concerns,

likes/dislikes, etc.

32

AssessmentReport No. 25

Dimensions of the Undergraduate Academic Experience: Results from the UAlbany Student Experience Survey

___________ 2006

Office ofInstitutionalResearch

http:/ /www.albany.edu/ ir/ reports.htm