Ed Tech 505 DSIS Math Tutorial Evaluation

5/19/2018 Ed Tech 505 DSIS Math Tutorial Evaluation

1/17

DSIS Math Tutorial Evaluation

Submitted to: Davis School for Independent StudyCompleted by: Christine Simms

May 9, 2014


2/17

DSIS Math Tutorial Evaluation|Submitted to: DavisSchool for Independent Study

Page 1 of 16

SummaryThe Math Tutorial Program at Davis School for Independent Study (DSIS) is an intervention thatwas put in place as a result of declining test scores and low course grades. It supplements thedirect instruction that occurs in weekly math classes to support improved academic achievement.It offers four hours a week of tutorial time staffed by the sites credentialed math teacher. The

tutorial hours give students opportunities to get help with their homework and clarify contentunderstanding through one-on-one interaction with the teacher. Student attendance is voluntaryuntil a students grade falls below a C-; at that time, the student is mandated to attend at least onehour of tutorial a week.

This formative evaluation was completed during the fourth year of the program to determine theextent to which it is meeting its established objectives. It presents an analysis of data gatheredfrom interviews with administrators and staff, standardized test scores, course grades, the 2011WASC Report, School Site Plans, program observations and a student survey as they relate tothe achievement of program goals to support students and increase academic achievement.

The evaluation found that the Math Tutorial Program is at least a contributing factor to student

success, but that academic achievement in mathematics remains a critical area of concern as datadoes not reflect the desired student outcomes. CST scores have not risen by the 5% a year goalsoutlined in the School Site Plans and many students are struggling to earn passing grades in theirmath courses. The report identifies areas where historic data is missing and suggestsimplementing tracking protocols that would support future evaluations.

Description of the program evaluatedDSIS is a K-12 site located in a small city flanked by a major university and surrounded byfarmland. Supervising teachers meet weekly with their students individually. Secondary mathcourses meet three times weekly (except Junior High Math which meets twice) and are taught bya credentialed math teacher. Because of the more limited contact time, math is a particularly

challenging subject to teach in an independent study setting.

The Math Tutorial Program was instituted in the 2010-2011 school year as one of the changes tothe structure of the math program after the schools Self-Study for their Western Association ofSchools and Colleges (WASC) accreditation renewal identified declining California StandardsTest (CST) scores in mathematics and low grades in math courses as critical deficits. Four hoursweekly FTE, divided across all five school days and distributed between morning and afternoonhours in order to accommodate the diversity of student schedules, is allocated to staffing thetutorial hours with the schools credentialed math teacher to better support student success in

mastering the course content delivered during math classes. (See Appendix A) This interventionprovides a less structured learning environment where students can get individualized help with

homework assignments, make up missed tests, and ask questions that support their developingunderstanding of concepts and processes.

The program serves all eight levels of secondary math courses taught at the site: Junior HighMath, Transition to High School Math, Pre-Algebra, Algebra 1, Geometry, Algebra 2,Trigonometry, and Business Math. It is open to all DSIS students on a drop-in basis and meets inthe school library. Once-weekly attendance is mandatory for students with Ds or Fs. The math


3/17


Page 2 of 16

teacher circulates among the students, checking in to see how things are going and responds toindividual questions as they arise.

Program ObjectivesThe Math Tutorial Program is in place to improve student achievement in mathematics at DSIS.Based on the programs goals, the following program objectives were established.

Students with grades of D+ or below will receive the individual support they need toincrease their overall course grades to passing by attending a minimum of one tutorial perweek until their grades improve.

Students will have the individual help they need outside of class to have their questionsanswered and content clarified to support content mastery as measured by increasingoverall course grades.

Students will receive the individual attention needed outside of class to have questionsanswered and content clarified to improve content mastery to proficient or above asmeasured by the CST.

Students will develop the content mastery necessary to increase confidence andengagement to prevent them from dropping math courses without completing them.

Students will receive the individualized attention they need to increase content masteryand confidence necessary to support them in continuing to progress on the math pathwaythrough high school graduation.

Program ComponentsThe foundation for Math Tutorial Program is four hours of dedicated FTE for the credentialedmath teacher to provide individualized support in a casual group setting, on a drop-in basis, forstudents taking all levels of math at DSIS. Students typically bring their own math books andgraphing calculators, but they are available to borrow as needed.

Evaluation MethodIn order to determine the extent to which Math Tutorial is meeting its established objectives, agoal-based model was used to design this evaluation.

ParticipantsGiven the relatively small size of the school, all math students in grades 7-12 were surveyedduring their math classes and students attending tutorial were observed as a part of thisevaluation. Additionally, interviews were conducted with the DSIS math teacher and two formerprincipals who had been directly involved in establishing the program.

Procedures

The evaluator began by meeting with staff and administration to determine program history,goals, and objectives, and identify existing data sources to be reviewed for the evaluation.Existing data was gathered for review of program history and student achievement since the yearprior to the institution of the program to establish a baseline and examine progress towardachieving program objectives.

A student survey was designed and administered as a follow up to the student survey on programparticipation completed in the spring of 2013 as well as collecting additional data related tostudent attitudes and future plans for continuing on the math pathway. Observations of Math


4/17


Page 3 of 16

Tutorial were conducted to see the program operating under normal, everyday conditions. It wasconfirmed with the math teacher that the observations during the evaluation period were typicalof normal use.

Finally, survey and academic achievement data were converted to excel for analysis and the finalevaluation report was produced for the school.

Data SourcesData was compiled from numerous sources to develop a comprehensive overview of theevolution of the program from its inception to current practice. The evaluator conductedinterviews with the DSIS 2010-current math teacher, Jamie Wales, 2012 Math Committee Chairand 2012-2013 principal, Laura Juanitas, and 2008-2012 principal, Kim Wallace. She reviewedthe 2011 WASC report, 2012 Math Committee meeting notes, School Site Plans for 2011-12,2012-13, and 2013-14, and math department syllabi.

Student achievement data was gathered from the 2011 WASC Accreditation Self-Study Report,School Site Plans, 2013-14 class marks recorded in Q, and 2013 California Standards Test (CST)

scores reported on Illuminate. Students also self-reported average grades on the student survey.

Data related to student understanding of and program use was gathered using a student survey(See Appendix B), evaluator observations, and previously utilized attendance trackingmechanisms/records. (See Appendix C) Existing data from a survey administered in the spring of2013 was also reviewed.

Student attitudes about math and future plans to continue on the math pathway were alsogathered using the student survey.

ResultsAcademic Achievement

Prior to changes made to the structure of math delivery at DSIS and the institution of MathTutorial, student achievement data was indicating a steady decline in the percentage of studentswho were performing at levels of proficient or above on the CST. (Table 1)

Though the state goal is for all learners tobe proficient or above, the 2010 resultsreported that 43.38% of DSIS studentswere in the proficient/advanced range.This prompted changes to the frequency

of math classes and the inception of theMath Tutorial Program. Moving forwardfrom this point, there was a noticeableupturn in CST score data.

Table 1


5/17


Page 4 of 16

Scores from the spring of 2011 indicated that the percentage of students scoringproficient or above had risen to 47% andthose from 2012 were fairly consistent withthat at 46%. There was a significant declinein the 2013 results, with only 30% of students

achieving proficient or advanced. (Table 2)

Course grade data available was limited tothe current year, 2013-14. At the end of thefirst semester, 49% of all math course gradeswere in the A/B range, 33% were Cs, and18% were Ds or Fs. Progress marks for thesecond semester are recorded at the end ofthe third quarter. At that time, almost thesame percentage (48%) of students wereperforming with sufficient mastery to earnAs or Bs, 14% of students were in the C

range, and 38% of student marks were eitherDs or Fs. (Table 3)

Survey DataThe student survey provided data related to participation in theMath Tutorial Program as well as information about studentattitudes about mathematics and their long-term math goals.

Of the students who completed surveys in 2014, 56% reportedattending Math Tutorial at least once during the current schoolyear. (Table 4) 14 of 52 students said that there are times whenattendance at Math Tutorial is mandatory. Of those, only 6indicated that the condition that would mandate attendance ispoor grades. None of the students who identified failing grades

2009 42.00%

2010 43.38%

2011 47.00%

2012 46.00%

2013 30.00%

0%

10%

20%

30%

40%

50%

60%

70%

80%90%

100%

CST Mathematics 2009-2013

Percent of Students Scoring

Proficient or Advanced

Table 2

A/B

49%C

33%

D/F

18%

2013-14 Semester 1

Math Grades for All Courses

A/B C D/F

Table 3

A/B

48%

C14%

D/F

38%

2013-14 Quarter 3

Math Grades for All Courses

A/B C D/F

Yes

56%

No

44%

2013-14 Student Use

of Math Tutorial

Table 4


6/17


Page 5 of 16

as triggering anattendancerequirementreported averaginggrades lower than

Cs.The top threereasons studentsreported choosing toattend Math Tutorialwere that theyneeded help with

homework, they didnt understand something in class, or a parent required them to come. (Table5) The three most frequently stated reasons for not attending were that they didnt need help, hadschedule conflicts, or that they get help elsewhere. (Table 6)

37% of studentssurveyed reportedhaving sourcesother than MathTutorial to supporttheir success inmath classes: 22 gethelp from parents, 8have UCD tutors, 7have other tutors, 7get help from

friends, and 4 have other sources of support. 76% of students reported that they always orusually understand content the content clearly enough to be successful by coming to class andcompleting homework/studying on their own.

A similar number of students weresurveyed in 2013. At that time, 63% ofresponses indicated having attendedtutorial at least once during the schoolyear. Of the students who were notelecting to attend, 58% cited scheduleconflicts and 21% reported that theyget help elsewhere.

The 2014 student survey alsogenerated baseline data related tostudent attitudes about math,themselves as math students, and their

plans for taking future math courses. (Table 7) The majority of students surveyed reportedfeeling pretty good at math and that they plan to continue taking math courses until high schoolgraduation. Students not only care about their math grades, but want to do even better.

1

0

4

65

13

20

3

11

0 5 10 15 20 25

Other

Help with Graphing

Like the Routine

Helps Me Feel ConfidentLike One-on-One Support

Didn't Understand

Needed Help with

Ms. Wales Required

Parent Required

Why Students Report They Attend Tutorial(could choose multiple responses)

Table 5

6

6

2

2

1

3

1

12

4

4

2

4

3

1

17

18

3

10

14

10

8

13

20

10

11

8

17

10

4

4

33

27

25

19

32

ENJOY MATH

GOOD AT MATH

CONTINUE MATH TO GRAD

CARE ABOUT GRADES

MATH IS IMPORTANT

TRY MY HARDEST AT MATH

WANT TO DO BETTER

Student Attitudes on a Scale of 1-5

1="no way" 5="definitely"

1 2 3 4 5

Table 7

3

9

3

9

24

19

23

0 5 10 15 20 25

Other

Might Help, but Not Motivated to Go

Don't Think it will Help

Satisfied with my Grade

Don't Need Help

Get Help Elsewhere

Schedule Conflict

Why Students Report They Do Not Attend Tutorial(could choose multiple responses)

Table 6


7/17


Page 6 of 16

Attendance DataTwo attendance tracking mechanisms were included in the review of existing data. In the 2010-2011 year, supervising teachers received an email listing all students who should be attendingtutorial (or working with a UCD tutor) due to low math grades. Students and their parents hadalready been notified; the idea was that supervising teachers could remind their students when

they saw them at their individual meetings in order to improve student follow-through. In the2012-2013 school year, the math teacher tracked attendance each week in a binder, writing downthe names of students present at each tutorial session and also collecting data related to theirdegree of engagement during the hour.

Policy Related DataSyllabi received by parents and students at the beginning of the course outline the tutorial policyas being optional for students with grades of C- or better, but attendance is mandatory at leastonce a week for those with current grades of D+ or below. The penalty for not attending when itis required is a warning letter. (See Appendix D)

Both administrator interviews indicated that attendance is mandatory for students with Ds or

Fs. Interviews with the math teacher offered the same data, but revealed that there is currentlyno tracking mechanism in place because the things she has tried have been cumbersome yetunsuccessful. Responses on the student survey revealed that while 27% of students believe thattutorial attendance is sometimes mandatory, only 12% are aware of low academic performancebeing the condition that triggers mandatory attendance.

DiscussionThe purpose of this evaluation was to determine the extent to which the Math Tutorial Programis meeting its established objectives. After several interviews with stakeholders and a thoroughexamination of all available data, it seems evident that the Math Tutorial Program is acontributing factor in the overall success of DSIS math students. However, given that at least two

major adjustments were made at the same time, an increase in the number of hours each mathclass meets weekly and the institution of Math Tutorial, along with incomplete historic data forcomparison, it is impossible to determine the precise impact of the program.

In looking at the CST scores, there is a significant increase (+3.62%) in the percentage ofstudents scoring proficient or above in the first test cycle (2010-2011) after the tutorial programwas established. There is a drop of 1% the following year, but given the small size of the school,this is insignificant and both the 2010-11 and 2011-12 school years present as consistent. The2012-13 scores offer a different picture, with only 30% of students scoring in theproficient/advanced range.

A number of possibilities that have nothing to do with Math Tutorial, or the math program atDSIS, may account for the sudden dip in scores. Scores are reported based on students currentlyenrolled at DSIS rather than on students who took the test at DSIS last year. Because the schoolstarted the year with fairly low enrollment, many of the current students were not at the site lastyear, thus their CST scores do not reflect on the teaching or interventions available at DSIS.Additionally, even students who take the CST at DSIS may not be receiving their instruction atthe site. Split site students may take math at a comprehensive site, but because they areconsidered DSIS students, they are tested at and scores are attributed to DSIS. Another


8/17


Page 7 of 16

possibility is a changing student population at the site. These scores should be further analyzed todetermine whether the decrease in scores is related to an increase in the population of theSocioeconomically Disadvantaged (SES) student subgroup. Historic data indicate that this grouphas performed significantly lower than the general DSIS population. If a change in population isresponsible for the dip in scores, it may be necessary to reconsider the best interventions for

meeting student needs.

Undoubtedly, academic achievement in math continues to be an area of concern at DSIS. CSTscores have failed to reach the 5% annual gains established in the school site plans and manystudents are receiving Ds or Fs in their math courses. Math Tutorial exists as an intervention tosupport student success, but is not being fully utilized to reach its potential. This may be due to agap in student perception. 10% of students surveyed self-reported D/F grades three weeks afterQ3 Progress marks indicated 38% of students were earning Ds or Fs.

Student survey data also indicated a lack of student understanding regarding the Math Tutorialattendance policy related to course grades. Though clearly stated in each course syllabus, only12% of students reported being aware of D/F grades making tutorial attendance mandatory.

Student confusion may be attributed to a failure to attend to that detail in the syllabus or may berelated to the disparity between the stated policy and current practice which does not trackattendance or penalize for lack thereof. The math teacher reported in her interview that she feelsthis is the biggest issue related to Math Tutorial. She has tried different methods, but has foundthem to be unsuccessful. She would like support in identifying a system that would work for bothher and the students, 44% of whom cite schedule conflicts as preventing them from attendingtutorial hours.

The 56% of students were have attended tutorial this year were well-served by the program. Farmore (60%) responses indicated reasons for attending related to seeking one-on-one support thanindicated attendance because a parent or the teacher required it (22%). This demonstrates a

conscious recognition of the value students place on having individual attention from the teacheroutside of class time.

67% of students report that they get help with their math from sources other than Math Tutorial.It may be appropriate to more closely examine the correlation between individual studentachievement and what sources they are using for support. Given the high percentage of studentswith low course grades and the number of students stating schedule conflicts as their reasons fornot utilizing the tutorial hours, it may be possible to provide more consistent, reliable support iftutorial hours can be configured differently or delivered in a different format, perhapssynchronously online.

Several program objectives relate to student attitudes about math as they affect decisions to drop

courses without completing them or long-term plans to continue progressing along the mathpathway beyond requirements for graduation. As there was no historic data in this area, it is notpossible to determine how the individual attention students receive as a result of Math Tutorialhas influenced changes in attitudes or plans. Additionally, there are no mechanisms in place totrack either course retention or how many/when/why students elect to stop taking courses inmathematics. If the objectives are to remain the same, it will be necessary to determine a plan forgathering the necessary data. The data collected for this evaluation may serve as a baseline in thearea of student attitudes for future evaluations.


9/17


Page 8 of 16

Table 8Objective Identified Strengths Identified Concerns

Students with grades of D+or below will receive theindividual support they needto increase their overallcourse grades to passing byattending a minimum of onetutorial per week until theirgrades improve.

Support in place to servestudent need

Attendance policy for D/Fgrades

Support underutilized by this targetaudience

No effective mechanism in place fortracking or enforcing attendancerequirement

Communication of policy to studentsnot clear

Students will have theindividual help they needoutside of class to have theirquestions answered andcontent clarified to supportcontent mastery as measuredby increasing overall coursegrades.

Students who attend aregetting individualizedsupport to clarify content

No tracking mechanism in place tocompare course grades historically

Students will receive theindividual attention neededoutside of class to havequestions answered andcontent clarified to improvecontent mastery to proficientor above as measured by theCST.

Support mechanism is inplace

Though growth was indicated, it neverreached 5% annually and declinedconsiderably in 2013

Further evaluation of the CST data isnecessary to address possible reasonsfor the decline

Students will develop thecontent mastery necessary toincrease confidence andengagement to prevent themfrom dropping math courseswithout completing them.

Most students have generallypositive attitudes about math

Some students who attendtutorial report that they do sobecause it increases theirconfidence

No historic data is available todetermine the role of programparticipation in student attitudes. Thisevaluation collected baseline data onstudent attitudes that can be used goingforward.

There is no readily available datarelated to course completion or thereasons students have dropped mathcourses

Students will receive theindividualized attention they

need to increase contentmastery and confidencenecessary to support them incontinuing to progress on themath pathway through highschool graduation.

Students report feeingsupported by the individual

attention they receive attutorial

Most students have generallypositive attitudes about math

Most students are planningto continue taking mathcourses until graduation

No historic data has been kept related toDSIS math students continuing to take

math until graduation. In order tofollow up on the achievement of thisobjective, it would be necessary toinstitute a mechanism for tracking thisinformation.

This evaluation collected baseline datarelated to student confidence that can beused for comparison in the future.


10/17


Page 9 of 16

The strength of the Math Tutorial Program is clearly in its ability to provide students with theindividual attention they need to address their own gaps in understanding. Because of thestructure of math instruction at DSIS, with the teacher having only two or three hours to deliverall of the content students will need to understand for the week, classes tend to be packed withdirect teaching and offer very few opportunities for student questions. Math Tutorial definitely

fills the gap for students by offering personal support and encouragement on an as-needed basis.

The weakness seems to emerge from the inability or lack of desire for some students to recognizetheir need for additional support, or from the schedule conflicts that prevent them from accessingthe resource. For this reason, it is necessary to establish a means of enforcing the attendancerequirement for those students who need help but are not coming on their own. It should bemanageable for both students and teacher. Rather than constant reevaluation, perhaps just use thefirst of each month as the cut-off date to mandate weekly attendance until a reevaluation thefollowing month. This would put all courses on the same cycle. A tracking sheet could be createdas a simple check-off that is marked when the student attends once during the week, no need totrack attendance for those who are not mandated to attend or for multiple days for those who are.

In looking at each of the objectives established for the Math Tutorial Program, this evaluationhas identified both strengths and concerns that can be used to begin conversations between staffand administration to chart a path for program improvement. (Table 8) It will be necessary torevise program objectives in light of the move to Common Core Standards and the transitionfrom the CST to the Smarter Balanced Assessment (SBAC). Until baseline scores are availableafter the 2014-2015 school year, achievement goals should be tied to course grades.

Project CostPrincipal Evaluator, Christine Simms, was on the project for a total of ten days of billable hours.She spent two days on site at the school and district office conducting interviews with staff andadministration, conducting observations of the program, and gathering existing data. She created

a student survey to be administered by the math teacher, then compiled the results for analysis.The balance of time was spent reviewing and analyzing data and producing the final evaluationreport.

Personnel Budget

Christine Simms [Principal Evaluator]: 10 days @ $500/day $5000

Total Personnel $5000

Miscellaneous

Supplies [paper, ink, copying, etc.]: $100

Total Miscellaneous $100

Total Budget $5100


11/17


Page 10 of 16

Appendices


12/17


Page 11 of 16

Appendix A

2013-2014 Secondary Class Schedule with Math Tutorial Indicated


13/17


Page 12 of 16

Appendix B

MATH STUDENT SURVEY (please do not include your name on this paper)Math course ___________________________ My test grades are typically: [A/B] [ C ] [D/F]

Is attendance at Math Tutorial ever mandatory? [Yes] [No] If yes, when? _________________Have you attended Math Tutorial during the current school year? [Yes] [No]

If yes, what did you find valuable? (select all that apply)[Having a scheduled time to do my math] [Working with others from my class][Getting immediate feedback to my questions] [Other:]_____________________________

How frequently do you attend Math Tutorial in general? (select one)[2+ times/week] [1x/week] [every other week] [1x/month] [


14/17


Page 13 of 16

Appendix C

2010-2011 Mandatory Attendance Tracking


15/17


Page 14 of 16

2012-2013 Attendance Tracking Document


16/17


Page 15 of 16

Appendix D

Algebra 1 Syllabus/Course Policies 2013-2014


17/17


Page 16 of 16

Date post:	10-Oct-2015
Category:	Documents
Upload:	chrsitinesimms
View:	36 times
Download:	0 times

Ed Tech 505 DSIS Math Tutorial Evaluation

Documents