Open  Technology  Professional  Development  Evaluation  for  Bonneville  Joint  School  District              Evaluation  Report  -‐  Course  Project  

Weight,  Clifton   12/13/13   EdTech  505-‐4201  

Summary  

This evaluation report was prepared by Cliff Weight Evaluations for the Bonneville Joint School District. The main purpose of this evaluation is to determine the effectiveness of professional development in regards to technology in the Bonneville Joint School District. Specifically, this report was prepared to evaluate the newly developed program to provide open, self-guided, technology help classes to teachers in the district. This program was created with the goal of increasing the use of technology in the classroom. Bonneville Joint School District feels that completing this goal is vital in the 21st century classroom to increase student engagement and achievement.

This evaluation will help determine whether this class is meeting its objectives. It will also include recommendations based on the data gathered within the evaluation for future improvement and modifications. Steps that I took to complete this evaluation include observation of the program and administration of a survey to participants in the program. The primary evaluator also used their first hand experiences as an instructor in the program to provide insights for the evaluation report. Lastly, the evaluator sought feedback from teachers and other instructors who participated in this program.

The evaluation report indicates that teacher’s response to the program was generally positive. Most survey results and feedback indicate a positive response to the program. Teachers enjoyed the flexibility provided by the program and the ability to receive help with the areas in which they needed it most. The weakness in the program was in the amount of participation garnered by the program. When the program was created Bonneville School District expected word of mouth and continued encouragement to help increase participation each week. This did not turn out to be the case as participation essentially stayed the same throughout the program. I will discuss these findings more in depth throughout this report.

Description  of  the  Program  Evaluated  

This program was developed by technology integration specialists at Bonneville School District with the goal of providing support to teachers in order to increase technology integration in classroom instruction. The district believes that student engagement and achievement increase when technology is used regularly in the classroom. Bonneville School District also knows that technology represents a significant investment of time and money and they want to ensure that they receive a return on this investment. In general, the school district feels teachers also understand the benefits of using technology in the classroom, but they often do not get the support and training the need to become effective when using technology in their classrooms. These are valid concerns that the district wanted to be able to provide a solution for.

Program  Objectives   Besides the goals talked about above, here are some of the other objectives the district would like to achieve using this program:

1. Increase teacher confidence and skills when using technology 2. Increase use of technology by teachers in the classroom. 3. Ensure that technology is not going unused in classrooms due to lack of training and

support. 4. Provide open self-guided training where teachers can come and receive help with

whatever technology they need help with. 5. Provide teachers with an opportunity to receive a professional development credit as an

incentive to learn more about technology.

Program  Components  The training program was arranged to meet Mondays from 3-6 PM in the training lab in the

Bonneville School District technology building. Teachers could come any time during those hours for help. The school district has three instructors available to help teachers during this time with any matter they want help with. The school district also had several other experts on different technologies that worked in the technology building that could be brought into the class to help with the field of technology that they specialize in. This first iteration of the class began in October 2013 and will end January 6th, 2014. Teachers could come whenever they were able and leave when they needed to. Teachers interested in earning PD credit were required to attend for a total of 15 hours. They were required to log their time and what they learned during that time. Once they complete the 15 hours they have to write a one page summary describing what they have learned as well as how they have, or how they plan on, integrating more technology into their classroom as a result what they have learned in this class. Teachers are not required to take the class for credit if they do not want and they are not limited to the 15 hours if they would like to attend more.

Evaluation  Method  

  Partic ipants   All of the teachers who took the class were a part of the evaluation. All teachers that participated in this program teach in a K-12 public school environment. The district serves a little over 10,000 students and employs around 600 teachers. The school district only had 15 teachers who consistently came to the open-training. With this small a number it seemed appropriate and necessary to include feedback from as many of the participants as possible in the evaluation. The instructor’s observations and feedback were also included in the evaluation. The district technology coordinator was also kept updated on aspects of the program and his feedback and thoughts on the program were used in part for this evaluation.

  Procedures   This evaluation was conducted by collecting both qualitative and quantitative data. Observation and interview questions were used to collect qualitative data, and surveys were used to collect quantitative data. These methods fall in line with the decision making method of evaluation. This method is appropriate to help the district make decisions regarding the future use and changes to be made to the program.

  Data  Sources   For this observation, it was necessary to gather data in regards to several different aspects of the program to ensure the districts objectives were being met. Teacher attitudes, abilities, performance, and participation all needed to be measured. In order to collect information on this a survey of the programs participants was conducted and teachers were asked to self-evaluate. This survey included questions about the teachers use of what was learned in this class and how that learning effected their attitudes in regards to technology and whether this class was helping them increase technology use in their classrooms. The survey also contained information to gauge the teacher’s use of specific technologies and interest in those technologies. Finally the survey gave teachers the opportunity to provide feedback on what they did and did not like about the program and ways they thought the program could be improved. A copy of the survey is included in the Appendix.

Results  

For this evaluation some of the most important results from the survey have been compiled into the table below. There were 15 respondents to the survey. The survey used scale of agreement statements as well as open ended questions about the program.

0  

2  

4  

6  

8  

10  

12  

My  overall  experience  in  this  class  has  been  

good  and  I  would  recommend  this  class  to  others.  

I  am  a  tech  savvy  person.  

I  struggle  to  use  technology  in  my  

classroom  instrucMon.  

I  have  been  able  to  use  more  

technology  in  my  classroom  as  a  result  of  this  

class.  

This  class  has  helped  me  improve  my  technology  related  skills.  

The  instructors  have  given  me  the  help  I  need.  

Strongly  Agree   Agree   Neutral   Disagree   Strongly  Disagree  

The results of the survey are fairly positive. This indicates that the objective to increase the technology related skills of our teachers has been achieved. The survey also shows that the school district had teachers from a variety of skill levels involved in the class and of those, almost all of them felt that this class helped them improve their skills. The survey also indicates that there is still room for improvement. A third of the respondents did not indicate that this class helped them use more technology in their classrooms.

The results of the observations for this program and feedback that the instructors received were also quite positive and coincided with the results from the survey. Most teachers were able to get help that was needed and most expressed their appreciation for the class. Teachers also appreciated the laid back environment and 1 on 1 help they could receive. Teachers also felt that they are constantly having to spend time outside of class trying to figure out technology. Those teachers indicated that it was beneficial to be able to get credit for this time outside of class while receiving the help they needed when they got stuck. They also liked being able to come get help with little specific things like how to create a chart in Excel and make PowerPoint presentations. Normally they did not want to bother someone with these seemingly minor issues or they might feel embarrassed. This class helped give them a good environment to pose these types of questions. Participants in the program also expressed their hope that the program would continue next semester as well.

Discussion  

The results of the data collected were used to help determine the value of the program. The data was also intended to help answer questions about the program effectiveness and help in making decisions regarding its future. If the program is continued, the results of the evaluation data will also be needed to recommend how to improve the program for future use. The data gathered was sufficient enough to be able to accomplish this.

Based on the data collected the evaluator has determined that the program was able to meet the goals and objectives set out for the participants that took part in the program. The results of the survey show that these teachers had a positive experience in the class and made good strides toward increasing the use of technology in their classrooms. Teacher efficacy was raised as a result of this program. It is the recommendation of this evaluation report that the program be continued and that it continues evolve and find ways to improve its offerings.

The biggest area in which this program needs to improve was in the small number of participants. The 15 regular participants represent less than 3 percent of the number of teachers in the district. If the district would like to create widespread change in the amount of technology being used in classrooms throughout the district they are going to have to find a way to get more teachers to participate in this program or others like it. This amount of participation has been acceptable for a pilot, but there needs to be more for the district to better meet their long term objectives.

At the beginning of the program one of the instructors sent out a district wide email to promote the class. The class was also promoted through word of mouth. Technology leadership in each school was asked to pass on information about the class. The class was also mentioned in the monthly teacher newsletter. It is recommended that this program be more heavily promoted to teachers. There also needs to be some follow up to make sure that technology leadership at each school did indeed promote the class to their teachers. Many of the participants that took the class did so in order to get the professional development credit. The district should try and find other ways to encourage teachers to come take the class. Another recommendation to increase participation would be to seek additional feedback about the time and location of the training. Most participants indicated in their survey that they felt the class was at a good time and place. It may be necessary to seek feedback of teachers that did not participate and find out what reasons kept them from participating. If time and location are found to be a factor, it is recommended that it be changed or that a time and location be added if the demand exists.

Other recommendations based on feedback from teachers is to increase the availability of different software experts in the district. At times teachers were unable to be as productive due to there not being an expert available on what they needed that day. Another suggestion would be to keep the class open and self-guided, but to have different weeks that contain a more specific focus on particular software or software areas. That way more experts and technicians with knowledge in that area could be scheduled to attend that day and better help teachers.

The program should be something that is continued for a few more semesters and then reevaluated for success. It may take several sessions of the class for the district to be able to observe whether the overall attitudes and use surrounding technology in the classroom has improved enough to continue down this route of professional development. Surveys should be conducted at least twice a year to continue to gain feedback and recommendations on how to improve the class.

Another recommendation to improve the program would be to provide teachers with more physical materials in order to help them once they have left the class. Several teachers expressed interest in receiving pamphlets or cheat sheets to refer back to in order to help them remember what they have learned. Teachers also expressed interest in having instructors perform short tutorials or introductions to new tools and technology. Most teachers indicated that they don’t even understand what technology is available to them and what it can do for them. If instructors can offer short tutorials, or explore some of these tools and their uses, it will be helpful to teachers. This will also give teachers a reason to come back to later classes in order to get information and help on the new programs that are shown to them in these short tutorials and introductions.

 

 

Project  Cost  

An invoice for this report is included on the following page. The cost of the project is based on the number of days spent planning and conducting the evaluation as well as per diem for travel and cost of materials. The project landed within the expected budget and time allotted. There was no cost of travel due to the training lab being located where the evaluator works. Need for materials was minimal as the biggest expense needed was to cover the cost of printing the surveys for participants in the program. The daily rate is based on the standard rate for evaluation in the area and was negotiated during the evaluation proposal. Due to this evaluation being performed for a graduate course project these costs have been waived.

Cliff  Weight  Evaluations  274  W  2nd  N  Rigby,  ID  83442  Tel  208555555    Fax    

 

INVOICE  NO.  1   12.13.2013  

BILL  TO   SHIP  TO   INSTRUCTIONS  

Bonneville Joint School District No. 93 555 Any Street Idaho Falls, ID 83401

QUANTITY DESCRIPTION UNIT PRICE TOTAL

4.5 Days Spent on Evaluation $500 $2,250

0 Travel Cost NA $0

Materials NA $5

 

SUBTOTAL   $2,255

SALES  TAX   $0

SHIPPING  &  HANDLING   $0

TOTAL  DUE  BY  1.31.2014   $2,255

Thank  you  for  your  business!  

Appendix  TECHNOLOGY PD SURVEY We hope you have been enjoying the class. Please take a minute to let us know how you think it is going and how we can improve future offerings. Thank you!

Statement Strongly Agree Agree Neutral Disagree

Strongly Disagree

My overall experience in this class has been good, and I would recommend this class to others.

I would be interested in taking similar classes in the future.

I am a tech savvy person.

This class has helped me improve my technology related skills.

I struggle to use technology in my classroom instruction

I have been able to use more technology in my classroom as a result of this class.

The instructors have given me the help I need.

The instructors are knowledgeable in what I need help with.

The class is at a good time and day.

I have been able to use more technology in my classroom as a result of the support my peers have given me.

I regularly use My Big Campus.

I would like to learn more about My Big Campus.

I have created a web page and use it as a resource for my class.

I would like to learn more about creating web pages.

I use NEOs in my classroom every day.

I would like to learn more about the NEOs.

I use Accelerated Math in my classroom and feel like I know how to use it with fidelity.

I would like to learn more about Accelerated Math.

I use Accelerated Reader in my classroom and feel like I know how to use it with fidelity.

I would like to learn more about Accelerated Reader.

I like the concept of having open technology hours, where I can come and get help with anything.

What was your initial purpose for taking this class? Have you been able to fulfill this purpose?

What are some things you have liked about this class?

What are some ways we could improve this class?

Are you an elementary or secondary educator?

Additional Comments: