Challenges using Multimedia integrated within a Science Curriculum using a Classroom-Centered Design Approach

Rebecca Deutscher, Ph.D. Center for Research, Evaluation, & Assessment

Lawrence Hall of Science, University of California at Berkeley ABSTRACT In this study, multimedia that is integrated within a middle school science curriculum was evaluated. The goal of the project was to make the multimedia work better for students and teachers in the classroom. In this evaluation, logistics issues as well as the quality of the multimedia curriculum were examined. These factors affect schools likelihood of implementing technology in their classrooms. The curriculum that was evaluated was Full Option Science System (FOSS), a hands-on, inquiry-based science curriculum. Two groups of participants took an online survey: 539 middle school teachers and 22 trainers. The teachers were asked about their backgrounds, the technology at their schools, and their impressions of the FOSS multimedia. The trainers answered questions about their background as a trainer, and their experiences with the multimedia during trainings. As a result, one discovered many challenges and issues connected with using the multimedia. These challenges included technical issues, location of computers, best way to present the multimedia, training, and quality of the multimedia. By examining the challenges, developers can improve professional development, teacher manuals, and multimedia associated with science curriculum to make it more useful for teachers, administrators, and other educators. In future studies, we can then examine the impact that a richer multimedia can have on enhancing students’ learning experiences and science skills. SUBJECT/PROBLEM According to the National Center for Education Statistics (2008) close to 100% of schools have access to computers and 97% have access to the Internet. However, Becker (2000) found that there is a large diversity in regards to the type of access to computers and the Internet and also in the quality of use. According to Songer (2007), schools may have access, but the technology is not being fully integrated into the curriculum. Present focus and research should be more of an emphasis on the quality of technology rather than just looking at the numbers of schools that have computers. Arafeh, Levin, Rainie, & Lenhart (2002) found that students felt there was a large disconnect between using the internet at home versus at school because of the quality of access and the heavy filtering at school. In addition, the students felt the quality of internet-based assignments was not as good as Internet-type activities that they do at home. Many schools restrict their Internet access. Cuban (2001) discussed the reluctance teachers have in using computers because of logistic problems using computers, such as slow internet connections, server crashes, software issues, etc. In order, for schools to really use the technology, the logistic problems need to be minimized and the quality of the technology needs to be improved. According to the American Association of University of Women (2000), teachers critiqued the quality of educational software, the lack of professional development, and the computer technical assistance. In this study, multimedia that is integrated within a middle school science curriculum was evaluated. The goal of this project was to examine logistical issues and the quality of a

multimedia that is integrated within a middle school science curriculum. This evaluation used a classroom-centered design approach that involves focusing on more than the learners themselves (Smith & Reiser, 1998). This approach involves examining the broader learning context of the school community, including teachers, as well as school and district culture, to understand how the software is affecting the classroom context (Loh et al., 1998). The teachers have an important role in using the curriculum and that should not be ignored (Brown & Edelson, 1998). In this evaluation, data was collected from teachers and professional development trainers about the quality and logistical issues connected with the multimedia. The goal was to improve the multimedia quality for students and teachers in the classroom and also to learn how to better support the schools in using the technology. As a result of this evaluation, one discovered many challenges and issues connected with using the multimedia. These challenges affect how students and teachers use the multimedia within the classroom. By examining these challenges, changes to the multimedia can be implemented. For this study, the science curriculum that was evaluated was Full Option Science System (FOSS). FOSS is a hands-on, inquiry-based science curriculum that was developed for both elementary and middle school students. It covers a wide variety of science topics that meet the National Science Educational Standards and correlates with many state standards as well. A part of this curriculum involves integrated multimedia to help students understand the concepts. This study focused on FOSS middle school courses: Chemical Interactions, Diversity of Life, Earth History, Electronics, Force and Motion, Human Brain and Senses, Planetary Science, Populations and Ecosystems, and Weather and Water. DESIGN/PROCEDURE In this study, two groups of participants were surveyed: middle school FOSS teachers and FOSS trainers. The teachers presently use or in the past have used the FOSS middle school multimedia. Trainers are either former or current teachers, who were hired to do FOSS curriculum professional development with teachers. In this study, 5000 teachers were contacted by email and received information about the FOSS multimedia evaluation study including an online survey website. There were 539 teachers who participated in the study and were given the online survey. The survey has approximately 150 questions that were open-ended, forced choice, and multiple responses. The teachers were asked about their backgrounds, the technology at their schools, their experiences with the FOSS courses, and their impressions of the FOSS multimedia. When the teachers answered questions about their impressions about the FOSS multimedia, they first reported which FOSS course they felt they had the most experience with and were the most comfortable teaching. Based on the teachers’ responses to these questions, they received further questions about the multimedia based on which course they chose. For example, the teachers who had the most experience with Populations and Ecosystems would then receive a series of questions about the specific multimedia components in this course. The reason this was done is because each course has approximately 50 to 70 different multimedia components. It would have been much too long for the teachers if they have filled out information for all the courses. Also the teachers were more likely to be able to answer the multimedia questions with a course they were more comfortable teaching. The total completion time on the survey varied from teacher to teacher (depending upon how many questions they answered), but it took about an hour.

FOSS Trainers were contacted by email. The trainers received information about the FOSS multimedia evaluation study including an online survey website. Twenty-two trainers

participated in the study. The survey had 30 questions that were open-ended, forced choice, and multiple responses. The trainers answered questions about their background as a trainer, their experiences with FOSS courses, their experiences doing trainings, their experiences with the multimedia during trainings, and their impressions of the teachers using the multimedia. The survey took about 15 minutes to complete. ANALYSES AND FINDINGS The teachers who participated represented 43 states and Puerto Rico, reflecting a diversity of responses from across the country (Figure 1). The largest percentage of teachers’ responses came from Washington, Pennsylvania, Kentucky, and New Jersey (between 8% and 10% each). About 2.5% of the teacher respondents were from 9 countries outside of the United States. All of the teachers were asked whether they taught in urban, rural, or suburban areas. The results indicated that 46% of teachers taught in urban areas, 32% taught in suburban areas, 17% taught in rural areas, and 4% said they were not sure.

Figure 2 shows the results of the home states of the FOSS trainers. The largest group came from Washington State (14%). In most of the states represented there were about two trainers. The trainers are fairly spread out around the country except in the southwest or in Texas where there are no FOSS trainers who took the survey. In addition, there were no trainers outside of the United States.

The teacher survey included questions related to the technology at their school (Figure 3). Windows desktop computer usage was reported at 65%, with Macintosh Desktop and Windows Laptop usage reported at 13%. Nine percent of the teachers reported using Macintosh Laptop computers. In this study, schools used Windows-based computers much more often than the Macintosh computers. A few teachers reported using DOS machines, hand-held devices, and Thin Clients.

The teachers were asked the age of their school computers (Figure 4). The responses included: 40% of teachers said they were two to three years old and 33% said they were 4 to 5 years old. There were about 13% of teachers who identified their computers as 8 or more years old.

The teachers were asked about Internet use and how they access the FOSS multimedia. In this evaluation, 98% of the teachers reported that students had Internet access at school. Teachers were asked how they access the FOSS multimedia (Figure 5): using the curriculum-provided CD’s, using the online multimedia, or using both? Twenty-one percent of the teachers use only the CD’s, 34% use only the online resources, and 45% use both CD and the online resources. Several teachers commented that they did not know that the multimedia was available online.

The teachers were asked a general question about access to technical support at their school. Eighty-nine percent reported that their schools had technical support. It is unknown, however, what kind of support this represents. Support can range from full-time technology staff to a staff teacher with some knowledge of computers. Teachers discussed some of their frustrations using the multimedia: CD’s that did not work, inability to download information, difficulty accessing the multimedia online, trouble with firewalls/pop-up blockers, difficulty using Flash/QuickTime, problems at log in, bandwidth problems, the multimedia is too slow, and specific multimedia did not work at all. The teachers were asked a series of questions about where they were most likely to use computers (classroom, mobile laptop cart, computer lab, library, media center, or other) and the number of students per computer. See Figure 6. Teachers could select multiple locations. The largest percent of teachers said they use computers in their own classrooms (43%), the second largest group used computer labs (40%), and the third largest was the mobile laptops (25%).

Based on the teachers’ responses to the question about where they were using computers (classroom setting, mobile laptops cart, computer lab, media center, library, or other location), they were asked further questions about the student to computer ratio in that setting (Figure 7). A specific breakdown shows that for those teachers using the computer lab, 76% report that students have their own computer and 13% report that two students share a computer. Fifty percent of the teachers who used the media center, library, or other location reported a 1:1 student to computer ratio while 40% of the mobile laptop cart users reported a 1:1 student to computer ratio and only 25% of the teachers who used computers within the classroom setting reported a 1:1 student to computer ratio. The teachers reported the number of students who shared a computer with one other person in various settings: 42% of the mobile cart users, 28% of the media center users, 27% of the library users, and 25% of the classroom users. Furthermore, in the classroom, 24% of the teachers reported that three to four students shared a computer and 12% reported that five to eight shared a computer. Overall, it seems that many students have the opportunity to work one-to-one or two-to-one with computers, but it generally is not in the classroom. Out of classroom settings is where students get more of an opportunity to work closely with the computer.

For the mobile computer carts, schools range in how many teachers share a cart. The teachers who said they used a mobile cart were asked how many teachers share a laptop cart. Figure 8 shows these results. It appears that the largest group of teachers said that they shared a laptop cart with ten or more teachers (21%) and the second largest group of respondents said that the whole school shared a laptop cart (19%). This indicates that at many schools where laptops are used, teachers may have a difficult time accessing laptop carts since so many teachers must use the same carts. The results did indicate that 12% of the teachers have their own laptop cart and 10% share it with just one or two teachers.

When asked a question about how teachers use the multimedia with students (Figure 9), 84% of teachers responded that they use the multimedia as a teacher-led demo, 60% said they have students work in small groups on the multimedia, and 10% give the multimedia as homework. Related to the response about homework, the teachers were asked to estimate how many students have access to computers at home. Thirty six percent of the teachers reported that between one-half and three-quarters of their students have computers at home and 26% said that between one-quarter and one-half of their students have computers at home. Only 3% of the teachers said that students could checkout computers from school and bring them home.

The teachers and trainers were asked questions about their experience with the multimedia during training: 62% of the teachers were trained on the FOSS multimedia, 20% were not trained on the multimedia, and 18% reported no FOSS professional development. The trainers were asked how often they use the multimedia during professional development. The results indicated that 95% said that they discuss the multimedia at every training session. They were then asked about how much time they spend discussing the multimedia at these trainings (Figure 10). The responses were: 66% said they spend 16 to 30 minutes, 17% said they spend 31 to 45 minutes, and 11% spend less than 15 minutes. Some sessions are half a day and others last a few days, such information was not taken into account in this question.

In Figure 11, the results show how the trainers discuss the multimedia to school staff. The largest group of trainers said that they had small groups of teachers working together (85%) and also said they did trainer-led demonstrations of the multimedia (85%). Few of the trainers said they only discussed the multimedia verbally (about 15%). The trainers could give multiple responses in this question.

The teachers also had lots of feedback in regards to changes that could be made to the multimedia to improve the quality. The teachers gave feedback on whether they used the multimedia or not, whether they felt it had an impact on student learning, the usability of it, the Teacher Guide, the help features, and on any other comments they had about the multimedia. Some examples of feedback from one of the courses that was received by teachers included:

• Would like more links to NASA and other government websites for updated pictures of different locations

• Need more student-centered simulations • Would be nice if it met standards exactly and did not take so long to make modifications • The graphics seemed to old • Need to reduce teacher preparation time • Need to create useful guides that help get through the multimedia without worrying about

what the students are doing in regards to their behavior • Would like to have more on planets

In this paper, the details of these results will not be discussed. The details of these results were discussed with the staff at FOSS in order to improve the quality of their multimedia. Discussion The results of this evaluation pointed project leaders to five major areas where challenges occurred with the multimedia. Challenge 1: Technical issues surrounding multimedia A major challenge when using technology is the technical problems. These problems can frustrate teachers and schools to the point where they do not want to use it. A difficulty with addressing this challenge is that it is not always known whether the technical issues reside at the development end or at the school. Often the curriculum/multimedia developers do not hear or learn about many of the problems. The developers also do not know what kind of technology or

access many of their schools have in order to better learn how to handle these issues. In this study, teachers were asked about their technology and issues that were occurring with the FOSS multimedia, thus enabling the multimedia developers to appropriately address the issues that reside at their end. The age of the computers used by the teachers can have a large impact on how well the multimedia works. The multimedia will be slower or may not work very well on older computer systems, or the graphics software that is needed to run multimedia may not work. As a result, teachers who have older machines may have a less positive view of the multimedia since the programs will not run as well as on newer computers. Historically, FOSS multimedia was accessed only through CD’s. In the past few years, online access to the multimedia has been has been provided, thus allowing more people access to the multimedia. It is possible that many of the teachers who use only the CD’s may be doing so because they did not realize that the multimedia is available online. It is also possible that many teachers have limited online access. By having both available for educators, it allows more people to have access to the multimedia. Schools with better technology support could help teachers deal more successfully with some of the frustrations they expressed regarding use of the FOSS multimedia. The amount of computer support could influence teacher’s likelihood of using the multimedia. The technology support staff could help determine what the problem is and how to figure out solutions. Also they could help decide if this is problem at the development end and then communicate that information to the developers. It is also important that curriculum/software developers have good technical support that educators can easily access. They can help determine where the problem is occurring and make modifications that would increase the likelihood that the multimedia will work in that school’s environment. Learning more about teachers’ access to different kinds of technology, allows one to find new, creative ways to improve the multimedia to better incorporate schools technology. It also helps to better understand how to deal with various technical issues that may arise. Challenge 2: Location of the computers influences how the multimedia is used Based on the data, schools have different situations in regards to their computer access. By understanding where teachers primarily use computers for the FOSS multimedia software, it will be easier to design the multimedia to incorporate the various settings. There are many schools where teachers go into computers labs or only have access to limited computers in their classrooms. The large number of teachers using computer labs means that they have to leave their classroom to use the multimedia. Many teachers discussed how it is difficult using the lab because of having to sign up ahead of time. There was also the difficulty of transitioning the whole class to another location. Another important issue is how many teachers or classrooms share the various computer resources. Often with computer labs, libraries, media centers, and laptop carts, the whole school has to share the resources. Teachers said it was difficult to have access to the lab because it is in high demand. Another issue is the student to teacher ratio in various settings. It seemed that many students have the opportunity to work individually or in pairs with computers, but it generally is not in the classroom. Out of classroom settings is where students get more of an opportunity to work closely with the computer. Since there are fewer computers in the classroom, teachers and designers need to think of creative ways to give students the opportunity to use the multimedia in the classroom. In designing the multimedia, all of these issues need to be taken into account. It may be helpful to include techniques of working

within different computer environments for teachers or ways to revise the multimedia part of the curriculum to include various computer settings. Challenge 3: Teacher practices and the best method to teach different multimedia units The question is what influences the way the teachers use the multimedia? Many of them use it as a demo. Is it because of their school’s computer situation or does the presentation of the multimedia change based on the type of multimedia? Some of the multimedia works better as a demonstration while others work better as a hands-on experience for the students. The training guide does not go into detail about the best method of teaching particular multimedia components. However, when examining the multimedia there is clearly a better approach for some of the multimedia. For some of the multimedia, by presenting it as a demonstration, students do not gain the benefit. Students need the hands-on experience and a small group would be much more beneficial. This information needs to be clearly stated. Also teachers need to be presented alternative ways of doing the multimedia if they do not have access to enough computers. A few years ago some experimental programs were in place to allow students to checkout computers, which would allow the multimedia to be assigned as homework. However, this system had some problems and has not been widely adopted, as was seen in this study. Presently, there are some schools and even states that are purchasing a laptop computer for every student to use both at home and at school. Teachers need to be aware of their students’ computer situation before they start using the multimedia as a homework assignment. Also developers should discuss which multimedia would make good homework assignments and how to implement this in the teacher manuals. Related questions would be what is the best way of using the multimedia and can changes be made to better accommodate the teachers’ situations? Challenge 4: The structure of professional development related to the multimedia Many teachers first exposure to a curriculum is during professional development. Teachers’ exposure to the multimedia during training may have an influence on how teachers use the multimedia in the classrooms and whether they use it at all. During professional development, teachers are introduced to the curriculum and learn how to present the material to students. Many of the trainers did demonstrations or discussed the multimedia verbally to teachers even though this may not be how they want the teachers to use the multimedia. During professional development, when teachers are shown the multimedia in the way it would be best presented (i.e. small groups, individual, etc.), they are more likely to present it to their students in that format. The teachers are more likely to feel comfortable with the multimedia if they have gone through training about not only the material, but also how to present it. Challenge 5: Improve the quality of the multimedia One of the major reasons that technology is not being fully implemented in the schools is because of the quality. Schools, districts, and teachers need to view multimedia as good quality before they will implement it. When creating multimedia curriculum, it is important to get feedback about the quality from the customers who are using it. In this evaluation, a large group of teachers were asked detailed feedback about all the multimedia within the FOSS courses. All of this data was written up and discussed with the FOSS staff. This feedback from the teachers is being used as a way to help redesign or make changes to the multimedia. As a result of these changes, the quality of the multimedia will improve and more schools will likely utilize this technology in their classrooms.

CONTRIBUTION/CONCLUSION This paper examined both the teachers and the trainers’ perspectives about the FOSS middle school multimedia. An examination of this data, allows developers to gain a better understanding of how teachers use multimedia and the challenges and issues associated with using integrated technology as part of a science curriculum. A thoughtful consideration of the challenges allows developers to improve professional development, teacher manuals, and multimedia associated with science curriculum and make it more useful for teachers, administrators, and other educators. In future studies, we can then examine the impact that a richer multimedia can have on enhancing students’ learning experiences and science skills.

REFERENCES American Association of University of Women (2000). Tech-Savvy: Educating girls in the new

computer age. Washington, DC: American Association of University Women. Arafeh, S., Levin, D., Rainie, L., & Lenhart, A. (2002). The digital disconnect: The widening

gap between internet-savvy students and their schools. Pew Internet Life Project. Retrieved March 17, 2009 from http://www.pewinternet.org/Reports/2002/The-Digital-Disconnect-The-widening-gap-between-Internetsavvy-students-and-their-schools.aspx

Becker, H.J. (2000). Who’s wired and who’s not: Children’s access to and use of computer

technology. The Future of Children: Children and Computer Technology, 10(2), 44-75. Brown, M., & Edelson, D. C. (1998). Software in context: Designing for students, teachers, and

classroom enactment. In A. S. Bruckman, M. Guzdial, J. L. Kolodner, & A. Ram (Eds.), Proceedings of ICLS 98: International Conference on the Learning Sciences (pp. 63-69). Charlottesville, VA: AACE.

Cuban, L. (2001). Oversold and overused: Computers in the classroom. Cambridge, MA:

Harvard University Press. Loh, B., Radinsky, J., Russell, E., Gomez, L. M., Reiser, B. J., & Edelson, D. C. (1998). The

progress portfolio: Designing reflective tools for a classroom context. Proceedings of CHI 98: Computer-Human Interaction (pp. 627-634). New York: ACM Press.

Smith, B. K., & Reiser, B. J. (1998). National Geographic unplugged: Designing interactive

nature films for classrooms. Proceedings of CHI 98: Computer-Human Interaction (pp. 424-431). New York: ACM Press.

Snyder, T. D., Dillow, S. A., & Hoffman, C. M. (2008). Digest of education statistics 2007

(NCES 2008-022). Washington DC: National Center for Education Statistics. (ERIC Document Reproduction Service No. ED500670)

Songer, N.B. (2007). Digital resources versus cognitive tools: A discussion of learning science

with technology. In S.K. Abell & N.G. Lederman (Eds.), Handbook of research on science education (pp. 471-491). Mahwah, NJ: Lawrence Erlbaum Associates.