+ All Categories
Home > Documents > Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010:...

Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010:...

Date post: 16-Mar-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
25
Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 1 of 25 Introduction The sixth component of the OSA contains questions for the Social and Behavioral Sciences core. The first questions ask the participants about how they fulfilled their Social and Behavioral Sciences core requirement. The following table shows the number and percentage of participants that selected each response to the first question regarding where students took their core curriculum course. The number of participants selecting each response adds up to more than the 755 total participants because those who did not select “I took my core curriculum class in social and behavioral sciences at Tech” could select more than one of the other responses.
Transcript
Page 1: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 1 of 25

Introduction The sixth component of the OSA contains questions for the Social and Behavioral Sciences core. The first questions ask the participants about how they fulfilled their Social and Behavioral Sciences core requirement.

The following table shows the number and percentage of participants that selected each response to the first question regarding where students took their core curriculum course. The number of participants selecting each response adds up to more than the 755 total participants because those who did not select “I took my core curriculum class in social and behavioral sciences at Tech” could select more than one of the other responses.

Page 2: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 2 of 25

How did you complete your core curriculum requirement in Social and Behavioral Sciences?

Response N % of all

Responses % of all

Participants

I took an individual and group behavior core curriculum class through dual credit in high school.

44 5.7% 5.8%

I took an advanced placement social and behavioral sciences core curriculum class in high school.

30 3.9% 4.0%

I took a CLEP exam for social and behavioral sciences core curriculum credit.

5 0.6% 0.7%

I received transfer core curriculum social and behavioral sciences credit for a class that I took at another institution.

249 32.0% 33.0%

I took my core curriculum class in social and behavioral sciences at Tech.

450 57.8% 59.6%

Total Responses 755 100.0%

For the analysis in this report the 755 participants are divided into the “TTU” group and the “ELSE” group. The TTU group represents the 450 participants (i.e., 59.6%) who selected “I took my core curriculum class in social and behavioral sciences at Tech” and the ELSE group represents the 305 participants (i.e., 40.4%) who selected one or more of the other responses indicating that they took their core curriculum class in the Social and Behavioral Sciences elsewhere. The following pie chart shows this division of the sample.

The 305 participants in the ELSE group were also asked if the class they took outside of Tech counted for their core curriculum credit. Of the 305 participants who reported taking a Social and Behavioral Sciences course elsewhere, 10 (i.e., 3.3%) reported that they did not know if the course counted for their Social and Behavioral Sciences core curriculum credit and 295 (i.e., 96.7%) reported that the course did count for their Social and Behavioral Sciences core curriculum credit. The 295 participants who reported that the course taken outside of Tech did count for their Social and Behavioral Sciences core curriculum credit were also asked which one counted. The following table shows the number and percentage of the 295 participants who selected each response.

450 (59.6%)

305 (40.4%)

Social/Behavioral

TTU

ELSE

Page 3: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 3 of 25

Which one?

Response N %

A dual credit class. 40 13.6%

An advanced placement class. 19 6.4%

A CLEP exam. 2 0.7%

A class I took at another institution. 229 77.6%

I don't know. 5 1.7%

Total 295 100.0%

The system stores some data for each of the participants and so it was possible to identify the respondents who major in the Social and Behavioral Sciences. The following majors were identified and classified as Social and Behavioral Sciences majors: Anthropology; Community, Family, and Addiction Studies; Communication Studies; Economics; Exercise and Sport Science; Geography; Human Development and Family Studies; Health; International Economics; Mass Communications; Political Science; Psychology; Sociology; and Social Work. These participants represent the so-called “experts” in the sample. The following table shows that there were a total of 146 Social and Behavioral Sciences majors in the OSA sample. It also displays how many participants were in each of the Social and Behavioral Sciences majors.

Social and Behavioral Sciences Majors

Major Frequency Percentage

Anthropology 6 4.1%

Community Family Addiction Studies 3 2.1%

Communication Studies 7 4.8%

Economics 2 1.4%

Exercise and Sport Science 29 19.9%

Geography 2 1.4%

Human Development and Family Studies 25 17.1%

Health 1 0.7%

International Economics 3 2.1%

Mass Communications 2 1.4%

Political Science 15 10.3%

Psychology 33 22.6%

Sociology 11 7.5%

Social Work 7 4.8%

Total 146 100.0%

Page 4: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 4 of 25

Results The student learning outcomes for the Social and Behavioral Sciences are:

To identify and critique alternative explanations for claims about social issues and human behavior.

To recognize the appropriate methods, technologies, and data that social and behavioral scientists use to investigate the human condition.

To identify and appreciate differences and commonalities between cultures.

To demonstrate knowledge of the origins and evolution of U.S. and Texas political systems.

The first student learning outcome aligns well with questions 1, 4, and 5. The second learning outcome aligns well with questions 2, 3, and 4. The third learning outcome does not clearly align with any of the assessment questions. The fourth learning outcome aligns well with question 6 through 10. The Social and Behavioral Sciences portion of the assessment contains ten knowledge questions. These are shown below as a screenshot from the actual instrument. For analysis purposes, the answers were coded from 1 to 4 in the order they appear on the actual instrument. Attachment D shows how many times each answer choice was selected by the different participants for all of the Social and Behavioral Sciences questions.

Page 5: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 5 of 25

Page 6: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 6 of 25

Page 7: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 7 of 25

Social and Behavioral Sciences 1: The chart below shows the distributions of answers for the first question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 3 is the correct choice. It can be seen that the majority of both groups chose the correct answer. Overall, a few more people in the TTU group chose the correct answer as compared to the ELSE group (73.3% vs. 66.6%). This difference is statistically significant at the 0.05 level (see attachment A for details). This suggests that on average students who take their class for the Social and Behavioral Sciences core requirement at TTU do significantly better with this question than the students who take it elsewhere. Since the first question aligns with the first learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course at TTU may be meeting this learning outcome more than students who take their course elsewhere.

7.8%

16.0%

73.3%

2.9%

14.4%16.7%

66.6%

2.3%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

1 2 3 4

Social and Behavioral Sciences 1

TTU

ELSE

Page 8: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 8 of 25

Social and Behavioral Sciences 2: The chart below shows the distributions of answers for the second question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 1 is the correct choice. It can be seen that the majority of both groups chose the correct answer. Overall, a few more people in the TTU group chose the correct answer as compared to the ELSE group (74.7% vs. 73.1%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement at TTU do not do significantly better with this question than the students who take their Social and Behavioral Sciences class elsewhere. Since the second question aligns with the second learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course at TTU may be meeting this learning outcome similar to students who take their course elsewhere.

74.7%

10.2% 9.6%5.6%

73.1%

10.8% 11.5%

4.6%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

1 2 3 4

Social and Behavioral Sciences 2

TTU

ELSE

Page 9: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 9 of 25

Social and Behavioral Sciences 3: The chart below shows the distributions of answers for the third question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 2 is the correct choice. It can be seen that less than half of both groups chose the correct answer. Overall, a few more people in the TTU group chose the correct answer as compared to the ELSE group (40.0% vs. 39.0%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement at TTU do not do significantly better with this question than the students who take their Social and Behavioral Sciences class elsewhere. Since the third question aligns with the second learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course at TTU may be meeting this learning outcome similar to students who take their course elsewhere.

46.4%

40.0%

8.9%

4.7%

46.9%

39.0%

8.9%

5.2%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

50.0%

1 2 3 4

Social and Behavioral Sciences 3

TTU

ELSE

Page 10: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 10 of 25

Social and Behavioral Sciences 4: The chart below shows the distributions of answers for the fourth question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 4 is the correct choice. It can be seen that less than a fourth of both groups chose the correct answer. This could indicate a lack of understanding in this area. However, it could also mean that this question is too difficult or that answer choices are too similar. Overall, a few more people in the ELSE group chose the correct answer as compared to the TTU group (24.9% vs. 22.0%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement at TTU do not do significantly better with this question than the students who take their Social and Behavioral Sciences class elsewhere. Since the fourth question aligns with the first and second learning outcomes, this suggests that on average students who take their Social and Behavioral Sciences course at TTU may be meeting these learning outcomes similar to students who take their course elsewhere. Neither group performed well on this question.

20.9%

29.1%28.0%

22.0%

17.7%

33.1%

24.3% 24.9%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

1 2 3 4

Social and Behavioral Sciences 4

TTU

ELSE

Page 11: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 11 of 25

Social and Behavioral Sciences 5: The chart below shows the distributions of answers for the fifth question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 3 is the correct choice. It can be seen that less than half of both groups chose the correct answer. As with question 4, this could indicate a lack of student knowledge in this area or the question could have been too difficult. Overall, a few more people in the TTU group chose the correct answer as compared to the ELSE group (35.1% vs. 33.1%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement at TTU do not do significantly better with this question than the students who take their Social and Behavioral Sciences class elsewhere. Since the fifth question aligns with the first learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course at TTU may be meeting this learning outcome similar to students who take their course elsewhere.

21.6%

24.4%

35.1%

18.9%20.7%

26.9%

33.1%

19.3%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

1 2 3 4

Social and Behavioral Sciences 5

TTU

ELSE

Page 12: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 12 of 25

Social and Behavioral Sciences 6: The chart below shows the distributions of answers for the sixth question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 1 is the correct choice. It can be seen that just over half of both groups chose the correct answer. Overall, a few more people in the ELSE group chose the correct answer as compared to the TTU group (58.4% vs. 54.7%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement elsewhere do not do significantly better with this question than the students who take their Social and Behavioral Sciences class at TTU. Since the sixth question aligns with the fourth learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course elsewhere may be meeting this learning outcome similar to students who take their course at TTU.

54.7%

24.9%

4.9%

15.6%

58.4%

24.3%

3.6%

13.8%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

1 2 3 4

Social and Behavioral Sciences 6

TTU

ELSE

Page 13: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 13 of 25

Social and Behavioral Sciences 7: The chart below shows the distributions of answers for the seventh question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 2 is the correct choice. It can be seen that less than half of both groups chose the correct answer. The fact that so many students chose the same wrong answer (i.e., answer 1) might point to a problem area for the students or could indicate a problematic question. Overall, a few more people in the ELSE group chose the correct answer as compared to the TTU group (42.6% vs. 36.4%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement elsewhere do not do significantly better with this question than the students who take their Social and Behavioral Sciences class at TTU. Since the seventh question aligns with the fourth learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course elsewhere may be meeting this learning outcome similar to students who take their course at TTU.

38.0%36.4%

12.9% 12.7%

30.8%

42.6%

11.5%

15.1%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

1 2 3 4

Social and Behavioral Sciences 7

TTU

ELSE

Page 14: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 14 of 25

Social and Behavioral Sciences 8: The chart below shows the distributions of answers for the eighth question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 1 is the correct choice. It can be seen that around half of both groups chose the correct answer. Overall, a few more people in the ELSE group chose the correct answer as compared to the TTU group (53.4% vs. 47.3%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement elsewhere do not do significantly better with this question than the students who take their Social and Behavioral Sciences class at TTU. Since the eighth question aligns with the fourth learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course elsewhere may be meeting this learning outcome similar to students who take their course at TTU.

47.3%

25.3% 23.8%

3.6%

53.4%

25.9%

18.7%

2.0%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

1 2 3 4

Social and Behavioral Sciences 8

TTU

ELSE

Page 15: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 15 of 25

Social and Behavioral Sciences 9: The chart below shows the distributions of answers for the ninth question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 2 is the correct choice. It can be seen that a large majority of both groups chose the correct answer, which may mean that this question is too easy to be a good discriminator of Social and Behavioral Sciences knowledge. Overall, a few more people in the TTU group chose the correct answer as compared to the ELSE group (97.8% vs. 97.0%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement at TTU do not do significantly better with this question than the students who take their Social and Behavioral Sciences class elsewhere. Since the ninth question aligns with the fourth learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course at TTU may be meeting this learning outcome similar to students who take their course elsewhere.

0.9%

97.8%

0.7% 0.7%1.6%

97.0%

0.0% 1.3%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

1 2 3 4

Social and Behavioral Sciences 9

TTU

ELSE

Page 16: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 16 of 25

Social and Behavioral Sciences 10: The chart below shows the distributions of answers for the tenth question for participants who took their class for the Social and Behavioral Sciences core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). Answer 1 is the correct choice. It can be seen that over half of both groups chose the correct answer. Overall, a few more people in the ELSE group chose the correct answer as compared to the TTU group (64.9% vs. 58.9%). This difference is not statistically significant at the 0.05 level (see attachment A for details). This means that on average students who take their class for the Social and Behavioral Sciences core requirement elsewhere do not do significantly better with this question than the students who take their Social and Behavioral Sciences class at TTU. Since the tenth question aligns with the fourth learning outcome, this suggests that on average students who take their Social and Behavioral Sciences course elsewhere may be meeting this learning outcome similar to students who take their course at TTU.

58.9%

30.7%

1.8%

8.7%

64.9%

26.9%

0.7%

7.5%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

1 2 3 4

Social and Behavioral Sciences 10

TTU

ELSE

Page 17: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 17 of 25

Social and Behavioral Sciences Average: The table below compares the differences between TTU and ELSE when the results for all the questions are averaged (e.g., if a student got 8 out of the 10 questions correct, his score will be 8/10 = .80). Both groups have low means around 50%. The mean is slightly higher for students who took their core requirement for the Social and Behavioral Sciences elsewhere. However, this difference is not statistically significant at the 0.05 level. This means that on average TTU performs about the same as ELSE on the Social and Behavioral Sciences section of the OSA.

Core at TTU Core Elsewhere

N Mean SD N Mean SD T-stat P-value

Social/Behavioral Overall 450 54.0% 16.7% 305 55.3% 16.8% -1.039 0.299

The chart below shows the distributions of the average scores for participants who took their class for the Social and Behavioral core requirement at TTU (blue) and participants who took their class for the Social and Behavioral Sciences core requirement elsewhere (red). The distributions are similar, but it looks like there are a few more in the ELSE group answering more than half of the questions correctly and that there are a few more in the TTU group answering less than half of the questions correctly.

0.0%1.3%

2.2%

7.3%

18.7%

24.0%22.7%

13.1%

6.4%

3.8%

0.4%0.0% 0.7%

3.6%

6.2%

16.4%

21.3%

24.3%

15.7%

8.2%

3.0%

0.7%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Social and Behavioral Sciences Overall: TTU vs. ELSE

TTU

ELSE

Page 18: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 18 of 25

The table below shows a comparison of the average scores for the participants selecting each course option within the ELSE group (those selecting more than one course option were excluded from the analysis). The table includes the F value and P value for an analysis of variance comparing the means. Although the table shows information for all four ELSE options, the group with less than 15 participants (CLEP Exam) was excluded from the analysis of variance.

Elsewhere N Mean St. Dev. F-value P-value

Dual Credit 29 58.6% 15.5% 2.357 0.097

Advanced Placement 17 61.8% 21.0% CLEP Exam 2 60.0% 14.1% Another Institution 236 54.1% 16.9% Total 284 55.0% 17.1%

Based on the mean, students who took their Social and Behavioral Sciences course through advanced placement are the highest-performing group and students who took their course at another institution are the lowest performing group. The means are not significantly different at the 0.05 level though. While this might be due to the small sample size of some of the groups, it suggests that on average students who take their Social and Behavioral Sciences course elsewhere perform similarly on the Social and Behavioral Sciences section of the OSA regardless of where they took their Social and Behavioral Sciences course.

Page 19: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 19 of 25

The chart below shows the distributions of the average scores for those participants who are majoring in the Social and Behavioral Sciences (blue) and participants who are not majoring in the Social and Behavioral Sciences (red). The distributions appear fairly similar with a few more Non-majors answering more than half of the questions correctly. Overall, the Non-majors have a slightly higher average than the Majors (54.8% vs. 53.6%). This difference is not statistically significant at the .05 level (see attachment B for details). This means that on average the Non-majors perform similarly to the Majors on the Social and Behavioral Sciences section of the OSA.

0.0%1.3%

3.1%

6.4%

17.7%

22.1%23.1%

14.6%

7.5%

3.5%

0.6%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Social and Behavioral Sciences Overall: Majors vs. Non-majors

Majors

Non-Majors

Page 20: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 20 of 25

The chart below shows the distributions of the average scores for the female participants (blue) and the male participants (red). The male participants had higher overall scores than the female participants (mean of 56.8% vs. 53.0%). This is significant at the 0.05 level (see attachment C). This suggests that on average male students do better with the Social and Behavioral Sciences section of the OSA. It appears in the chart below that the significant difference in average scores of the female and male participants is due to more male participants answering more than half of the questions correctly and more female participants answering less than half of the questions correctly.

0.0% 0.5%2.5%

7.7%

19.5%

27.2%

23.1%

11.6%

4.8%

2.5%0.7%0.0%

1.9%3.2%

5.8%

15.1%17.0%

23.4%

17.9%

10.6%

4.8%

0.3%0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Social and Behavioral Sciences Overall by Sex

Female

Male

Page 21: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 21 of 25

The following table shows the correlations between the overall average for the Social and Behavioral Sciences questions and time to complete OSA, GPA, SAT score, ACT score, transfer credits, total credits earned, and age (p-values for the correlations are in parenthesis). The correlations with time to complete OSA, GPA, SAT score, and ACT score are significant at the 0.05 level. These correlations suggest that on average students who took less time to complete the OSA, students with higher GPA’s, students with higher SAT scores, and students with higher ACT scores do better on the Social and Behavioral Sciences section of the OSA. Some of these correlations are small and are more likely to be found statistically significant because of the large sample size.

Time GPA SAT ACT Transfer Credits

Total Credits Age

Social/Behavioral Overall

Correlation -0.103 0.154 0.404 0.281 -0.010 0.031 0.042

P-value (0.005) (<0.001) (<0.001) (<0.001) (0.792) (0.394) (0.248)

N 755 755 511 388 755 755 755

The following tables show the results of regression models for the overall average for the Social and Behavioral Sciences questions including all of the variables that have been explored in this analysis. There are three separate regression models because not all of the participants have an SAT score and ACT score. Since most students either have an SAT score or an ACT score, there would be too many missing values if both scores were included in the same regression model. The first model excludes both in order to include most respondents in the analysis.

Social/Behavioral Model 1

N F P-value

753 5.64 < 0.001

Variable Coefficient P-value

Intercept 0.375 < 0.001

Time -0.000030 0.006

Sex -0.043 < 0.001

GPA 0.049 < 0.001

Transfer Credits -0.00020 0.593

Total Credits 0.00016 0.582

Age 0.0020 0.091

Social/Behavioral Major -0.005 0.756

Social/Behavioral class taken at TTU -0.0210 0.161

This first model excludes SAT and ACT score to include 753 of the 755 participants. The model overall is significant at the 0.05 level (R2 = 0.0571). For this model time to complete OSA, sex, and GPA are the significant predictors at the 0.05 level for the overall average for the Social and Behavioral Sciences questions. These predictors suggest that on average students who take less time to complete the OSA, male

Page 22: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 22 of 25

students, and students with higher GPA’s do better on the Social and Behavioral Sciences section of the OSA when the other variables in the model are held constant.

Social/Behavioral Model 2

N F P-value

511 15.41 < 0.001

Variable Coefficient P-value

Intercept -0.127 0.128

Time -0.000039 0.006

Sex -0.024 0.085

GPA 0.017 0.186

Transfer Credits 0.00011 0.812

Total Credits 0.000082 0.811

Age 0.0071 < 0.001

Social/Behavioral Major 0.0145 0.398

Social/Behavioral class taken at TTU -0.029 0.081

SAT 0.00044 < 0.001

This second model includes SAT score and excludes ACT score to include 511 of the 755 participants. The model overall is significant at the 0.05 level (R2 = 0.2168). For this model time to complete OSA, age, and SAT score are the significant predictors at the 0.05 level for the overall average for the Social and Behavioral Sciences questions. These predictors suggest that on average students who take less time to complete the OSA, older students, and students with higher SAT scores do better on the Social and Behavioral Sciences section of the OSA when the other variables in the model are held constant.

Social/Behavioral Model 3

N F P-value

387 4.98 < 0.001

Variable Coefficient P-value

Intercept 0.101 0.317

Time -0.000022 0.597

Sex -0.031 0.076

GPA 0.024 0.168

Transfer Credits -0.00001 0.987

Total Credits 0.00023 0.578

Age 0.0052 0.05

Social/Behavioral Major 0.007 0.754

Social/Behavioral class taken at TTU -0.017 0.416

ACT 0.011 < 0.001

Page 23: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2010: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 23 of 25

This third model includes ACT score and excludes SAT score to include 387 of the 755 participants. The model overall is significant at the 0.05 level (R2 = 0.1062). For this model, age and ACT score are the significant predictors at the 0.05 level for the overall average for the Social and Behavioral Sciences questions. These predictors suggest that on average older students and students with higher ACT scores do better on the Social and Behavioral Sciences section of the OSA when the other variables in the model are held constant.

Limitations It is difficult to measure the breadth of knowledge taught in the Social and Behavioral Sciences with ten questions. Many of the questions seem to have challenged the participants. They could either point to a problem area for the students or they could indicate a problematic question. As mentioned in the results, a few of the questions may have been too easy to be a good discriminator of Social and Behavioral Science knowledge. That the Social and Behavioral Sciences majors did not do significantly better on average than everyone else suggests that the measure may not be capturing participants’ understanding of the Social and Behavioral Sciences too well. However, it might also be that a comparatively larger number of the Social and Behavioral Sciences majors did not take their time with the questions. In this case it would be more a reflection of the Social and Behavioral Sciences majors and not the questions.

Page 24: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2009: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 24 of 25

Attachments Attachment A: Summary of Chi-Square Tests for Questions 1 – 10

TTU (N=450)

Else (N=295)

correct incorrect correct incorrect Chi Statistic Chi Probability

Social/Behavioral 1 330 120 203 102 4.02 0.0449

Social/Behavioral 2 336 114 223 82 0.23 0.6332

Social/Behavioral 3 180 270 119 186 0.07 0.7863

Social/Behavioral 4 99 351 76 229 0.87 0.3511

Social/Behavioral 5 158 292 101 204 0.32 0.5707

Social/Behavioral 6 246 204 178 127 1.01 0.3155

Social/Behavioral 7 164 286 130 175 2.92 0.0876

Social/Behavioral 8 213 237 163 142 2.71 0.0995

Social/Behavioral 9 440 10 296 9 0.39 0.5305

Social/Behavioral 10 265 185 198 107 2.79 0.0951

Attachment B: 2-Sample T-Test for Average Scores of Majors and Non-Majors

Majors Non-Majors

N Mean SD N Mean SD T-stat P-value

Social/Behavioral Overall 146 53.6% 15.2% 609 54.8% 17.1% -0.789 0.430

Attachment C: 2 Sample T-Test for Average Scores by Sex

Female Male

N Mean SD N Mean SD T-stat P-value

Social/Behavioral Overall 441 53.0% 15.5% 312 56.8% 18.1% -3.106 0.002

Page 25: Online Senior Assessment 2009: Social and Behavioral Sciences · Online Senior Assessment 2010: Social and Behavioral Sciences Office of Planning and Assessment, Devin DuPree and

Online Senior Assessment 2009: Social and Behavioral Sciences

Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, July 2010 Page 25 of 25

Attachment D: Number of Participants Selecting Each Answer for Each Question

Social/Behavioral 1 Social/Behavioral 2 Social/Behavioral 3

Answer TTU ELSE Answer TTU ELSE Answer TTU ELSE

1 35 44 1 336 223 1 209 143

2 72 51 2 46 33 2 180 119

3 330 203 3 43 35 3 40 27

4 13 7 4 25 14 4 21 16

Social/Behavioral 4 Social/Behavioral 5 Social/Behavioral 6

Answer TTU ELSE Answer TTU ELSE Answer TTU ELSE

1 94 54 1 97 63 1 246 178

2 131 101 2 110 82 2 112 74

3 126 74 3 158 101 3 22 11

4 99 76 4 85 59 4 70 42

Social/Behavioral 7 Social/Behavioral 8 Social/Behavioral 9

Answer TTU ELSE Answer TTU ELSE Answer TTU ELSE

1 171 94 1 213 163 1 4 5

2 164 130 2 114 79 2 440 296

3 58 35 3 107 57 3 3 0

4 57 46 4 16 6 4 3 4

Social/Behavioral 10 Answer TTU ELSE

1 265 198 2 138 82 3 8 2 4 39 23


Recommended