Meta-Analysis of National Research Regarding Science Teaching

Prepared for

Texas Science Initiative of the Texas Education Agency Shirley Neeley, Ed.D., Commissioner of Education

September, 2005

Meta-Analysis of National Research Regarding Science Teaching

Texas A&M University Center for Mathematics and Science Education Project Staff: Timothy P. Scott, Ph.D. Homer Tolson, Ph.D. Carolyn Schroeder Yi-Hsuan Lee, Ph.D. Tse-Yang Huang, Ph.D. Xue Hu Adrienne Bentz Advisory Board: Carol L. Fletcher, Ph.D., Texas Regional Collaboratives, UT Austin Ginny Heilman, Region VI ESC Anna McClane, Region IV ESC Sandra S. West, Ph.D., Texas State University Jo Ann Wheeler, Region IV ESC Review Team: Katherine (Kit) Price Blount, Ph.D., Texas Collaborative for Excellence in Teacher

Preparation Patricia Castellano, North East ISD, San Antonio Diane Jurica, George West ISD Judy Kelley, Texas Rural Systemic Initiative & South Texas Rural Systemic Initiative Mayra Martinez, TAMU-Corpus Christi Sharon Kyles Ross, Dallas ISD Fernando Ruiz, TAMU-Corpus Christi

Texas Education Agency Project Staff: Robert Scott, Chief Deputy Commissioner Christi Martin, Senior Advisor Susan Barnes, Ph.D., Associate Commissioner Chris Castillo Comer, Director of Science Gina S. Day, Texas Science Initiative Manager Sharon Jackson, Ph.D., Deputy Associate Commissioner Irene Pickhardt, Assistant Director of Science George Rislov, Managing Director of Curriculum

An Introduction to the Texas Science Initiative

In 2003, the 78th Texas Legislature enacted HB 411 to improve science education at all levels and prepare Texas students for postsecondary success. Led by Governor Rick Perry, a community of education leaders and policy makers established a plan for the initiative, conducted a needs analysis of the current state of science education in Texas, and recommended action steps for responding to those needs with scientific research-based professional development, instructional strategies, and classroom materials. The resulting group of programs, known as the Texas Science Initiative, strives to address these challenges through a number of ventures, including the creation of professional development modules emphasizing effective strategies for teaching Science; online diagnostic instruments to aid teachers in student needs assessment; after-school and summer programs for struggling students; and the Master Science Teacher Certification Program. In order to aid the Agency and the Commissioner in the creation of training materials and other resources to assist science teachers in developing expertise in effective instructional approaches, the Texas Education Agency commissioned Texas A&M University at College Station to conduct a meta-analysis of national science education research to identify the most effective science instructional tools and methods. The purpose of this meta-analysis is to define what has been shown to improve student achievement and to develop a publication designed to share that information with educators across the state. This research provides the basis for production of a rubric for evaluating state-invested and other science education professional development and instructional materials to ensure the integrity and reliability of those products. The identification and implementation of proven methods of science instruction is a key step in ensuring the success of Texas students in science achievement and will inform future policy decisions and resource investments of science education stakeholders and policy makers.

Initial Report

Meta-Analysis of National Research Regarding Science Teaching i

CONTENTS

Introduction to the Meta-Analysis...………………………………………………... 1

Methodology………………………………………………………………………….. 3

Acquisition of Studies………………………………………………………...... 3

Coding of Studies……………………………………………………………..... 4

Intercoder Objectivity………………………………………………………....... 5

Criteria for Selection of Studies……………………………………………..... 6

Computation of Effect Size…………………………………………………..... 7

Statistical Methods and Analysis……………………………………………… 9

Results………………………………………………………………………………… 10

Section 1: Description of Studies…………………………………………....... 10

Section 2: Meta-Analysis Validity Issues…………………………………...... 16

Section 3: Meta-Analysis for All Studies…………………………………....... 19

Section 4: Meta-Analysis for Studies Classified by Treatment Categories. 22

Section 5: Analysis of Moderator Variables………………………………….. 27

Conclusions…………………………………………………………………………... 28

References………………………………………………………………………….... 30

Studies Included in Meta-Analysis…………………………………………………. 31

Appendix A. Web Sites and Search Terms……………………………………….. 35

Appendix B. Bibliography of Studies and Articles………………………………... 38

Appendix C. Literature Coding Document………………………………………… 63

Appendix D. Treatment Description Categories Classification Form….……….. 65

Initial Report

Meta-Analysis of National Research Regarding Science Teaching ii

Appendix E. Treatment Description Summaries………………….…………….... 67

Appendix F. Comprehensive Meta-Analysis Output …………………………….. 78

Appendix G. Regression Output……………………..…………………………….. 92

Appendix H. Formulas………….………………………………………………….... 96

Initial Report

Meta-Analysis of National Research Regarding Science Teaching 1

Introduction to the Meta-Analysis The Center for Mathematics and Science Education at Texas A&M

University has undertaken a research project commissioned by the Texas

Education Agency in order to improve the learning and academic performance of

students in K-12 science so that they are prepared for post-secondary success.

This portion of the research project, a meta-analysis of national research

regarding science instruction, was initiated to establish a knowledge base of what

methodologies have been shown to be effective and which ones have not been

effective. The meta-analysis addressed the question: What teaching

methodologies have been shown to improve student achievement in science?

While this question may appear too broad for a meta-analysis, the broadness was

intentional to encompass the breadth of science methodologies employed and the

multitude of instruments used to assess student achievement. The findings of this

study will be used to develop a publication to share this vital information with

educators across the state and to provide the criteria for evaluation of instructional

materials and professional development programs.

The methodology involved in this project consisted of six major phases:

acquisition of studies, coding of studies, determining intercoder objectivity,

establishing criteria for selection of studies, computation of effect size and

establishing statistical methods and conducting analyses. The study acquisition

phase involved searching for research reports as well as position papers,

bibliographies, and other relevant references and collecting as many pertinent

reports as possible for the meta-analysis. The first step of the study coding phase

consisted of developing a coding instrument to identify all of the necessary types

of information to be collected from each report in order to describe the

interventions used in the studies. Each report was then screened to determine if it

met the preliminary criteria for inclusion in the meta-analysis. Those studies

meeting the preliminary criteria were then carefully coded using the coding

instrument. The third phase, intercoder objectivity, involved the simultaneous

coding of three studies by three members of the research team. A listing of the

Initial Report


stringent criteria for final selection of studies was the fourth phase of the project.

The fifth phase of the methodology was the computation of effect sizes. The final

phase involved the use of Comprehensive Meta-Analysis® (CMA) software and

SPSS® statistical analysis software, to obtain results for display and interpretation.

This initial report is limited in scope due to the rigorous inclusion criteria

necessitated by the time limits for completion. It is highly recommended that the

following types of studies be included in future meta-analyses:

o International studies

o Correlational studies (data on two variables collected and

summarized, showing the relationship between the variables)

o Studies dealing with attitudinal and motivational changes in students

and teachers

o Studies dealing with special populations (English-language learners,

special education, under-represented populations, etc.)

o Studies dealing with teacher professional development

o Studies dealing with learning in general (not just limited to science

education)

Initial Report


Methodology Acquisition of Studies A broad search for reports of studies to be included in the meta-analysis

resulted in the collection of over 390 studies including journal articles, conference

papers, books, dissertations, government reports and unpublished papers. The

majority of the reports were identified through searches of electronic databases,

including ISI Web of Science, ERIC, ERIC EBSCO, ERIC FirstSearch, ERIC CSA

(Cambridge Scientific Abstracts), Academic Search Premier, PsycINFO, and

ProQuest Dissertations and Theses. Government web sites such as Berkeley

National Laboratory, the Department of Education, and other science education

sites such as the National Academies, National Science Resources, American

Association for the Advancement of Science (AAAS), Education Development

Center, and National Science Teachers Association provided reports, links to

sources, and names of science programs. In addition, general Web searches

using standard search engines such as Google and Google Scholar were

conducted. Initial search terms included “science education” and “student

achievement.” Subsequent searches were expanded by using various

combinations of alternatives for achievement, such as “performance,” “success,”

and “outcomes,” and by combining them with individual science disciplines such

as “biology,” “chemistry,” “physics,” “physical science,” “earth science,” “ecology”

and “environmental science.” Other terms such as “science teaching,” “hands-on

science,” and “professional development” were also used in search strings. (See

Appendix A for a listing of web sites and search terms employed.)

A request was sent by email to the National Association for Research in

Science Teaching (NARST) listserve soliciting suggestions of published articles or

reports that might contain useful research or for copies of unpublished research.

Experts in the field of science education research were asked to review the

bibliography and make suggestions. In addition, direct requests for information

were sent individually to project directors of “exemplary and promising science

programs” identified by the Department of Education Expert Panel on

Initial Report


Mathematics and Science Education. The programs contacted included the

Biological Sciences Curriculum Studies (BSCS), Project ARIES, Project InSIGHT,

Event-Based Science, Foundational Approaches in Science Teaching (FAST),

Full Option Science System (FOSS), Great Explorations in Math and Science

(GEMS), Modeling Instruction in High School Physics, Phenomena and

Representations for the Instruction of Science in Middle Schools (PRISMS), and

Science 2000+. Finally, reference lists from books, dissertations, studies and

other meta-analyses were examined for potential studies to be included.

Coding of Studies Characteristics of the studies (moderator variables) were coded in order to

investigate the possible influence of some of these variables on effect size. Coded

attributes included:

• study number and citation,

• publication type (refereed journal article, dissertation, unpublished report),

• study type (experimental – complete randomization, quasi-experimental –

randomization used, quasi-experimental – no randomization, correlational),

• dependent variable [type of test, test name, number of items, content

area(s)],

• independent variable (treatment name and description, control and/or

alternate treatment),

• length of treatment/study,

• setting and characteristics,

o schools (number of schools, if selected at random, if unit of analysis,

public/private, urban/rural/suburban, size, % free lunch),

o students (number of students, if selected at random, if assigned at

random, if unit of analysis, gender, grade, ethnicity, socioeconomic

status), and

o teacher(s) (number of teachers, if volunteer or selected,

age/experience, gender, certification,

Initial Report


• study results [effect size(s), p(s), t(s), F(s), eta square(s), omega

square(s)],

• study design classification,

o true random assignment of schools/students to treatment and

control groups,

o quasi-experimental with match of schools/students to achievement

and demographics of comparison school/group,

o quasi-experimental with covariate adjustment for prior achievement

differences,

o quasi-experimental comparison of schools/subjects based a claim of

“similarity,”

o quasi-experimental comparison of schools/subjects to regional,

state, or national data,

o quasi-experimental single-group pre-post comparison,

o quasi-experimental treatment vs. control pre-post test, or

o quasi-experimental multiple group ANOVA (Analysis of Variance).

Intercoder Objectivity

In order to establish intercoder reliability, three journal articles were

selected at random and coded independently by the senior analyst and two

members of the research team. For two of the articles, the degree of objectivity

was 90%. Due to the non-inclusion of correlational studies in this meta-analysis,

only two items were coded for the third article as the coders stopped after

identifying the study as a correlational study. The remainder of the articles for this

project was divided between the two members of the research team who coded

their respective articles and then submitted them to the senior analyst. The senior

analyst then read and coded all of the articles and any differences in coding

values were resolved at the discretion of the senior analyst. All of the dissertation

material was evaluated and coded by the senior analyst and two recent Ph.D.

recipients who possess strong statistical backgrounds.

Initial Report


Criteria for Selection of Studies

In the initial search for studies, liberal criteria were employed and many

reports, studies and articles were collected which proved to be unusable for this

meta-analysis. For final inclusion in the meta-analysis, rigorous criteria which

conformed to the charge from TEA were employed. Studies had to have:

• been published between January 1, 1980, and December 31, 2004,*

• been concerned with K-12 science education in the U.S.,

• used student achievement (or success, performance, etc.) as the

dependent variable,

• used science education teaching strategies as independent variables,

• been experimental or quasi-experimental,

• reported effect size (ES) or the statistics necessary to calculate ES (means

and standard deviations, p values, ANOVA tables, etc.),

• not been totally correlational,

• been for general education students/classes (not deal exclusively with a

special population), and

• not been included more than once (e.g., the same study reported in a

conference paper and a journal article).

An indication of the number of studies that were not included and the

reasons for non-inclusion are presented in Table 1.

*This time span encompasses the studies conducted since the major science education meta-analyses published in the early 1980s.

Initial Report


Table 1. Frequencies and Reasons for Non-Inclusion Reasons Frequencies

Did not fit time frame * Did not concern K-12 science education in the US 58 Did not use student achievement as DV and/or science education methodologies as IV 97

Was not experimental or quasi-experimental (cannot be correlational) 41 Did not have ES or necessary data to calculate ES 62 Was not for general education students (concerned with special populations) 25

Duplicated another study 6 Other (a meta-analysis report, bibliography, descriptive case study, position paper, literature review, etc.) 48

Total 337 *A large number of studies were not collected because it was immediately obvious that they did not fit within the time frame. Computation of Effect Size

For each achievement measure reported in the included studies, an effect

size (ES) was calculated comparing the performance of a treatment group with

that of a group that received a control treatment. Basic effect size calculations

were based on Cohen’s d (Cohen 1988) using the formula:

ScXXES CT −=

where d is replaced with ES and TX is the mean for the experimental or treatment

group, CX is the mean for the control group, and SC is the standard deviation of

the control group.

For studies which employed single group pre-post test designs, the effect

size was calculated as

pre

prepost

SXXES −

=

Where postX is the mean for the posttest, preX is the mean for the pretest, and

preS is the standard deviation for the pretest data. The above formulas were used

in hand calculations of effect size for many of the articles in this meta-analysis.

Initial Report


These previously cited indicators of effect size are presented as Hedges’ g

in the CMA® and involve the following formula:

P

CT

SXX

ES−

=

where ( ) ( )

211 2222

2

−+−+−

=CT

CCTTP nn

SnSnS which is the pooled variance.

In cases in which effect sizes had to be calculated indirectly, that is from t,

procedures recommended by Hedges and Becker (1986) were employed:

n

tES 2= (for equal ns) and

CT nn

tES 11+= (for unequal ns).

For the multiple groups ANOVA research designs, the transformation

formula used to convert F to ES was:

CT

CT

nnnn

FES+

=

where F is the obtained ANOVA test statistic and nT and nC represent respective

sample sizes.

If a sample size was less than 30, the standardized mean difference effect

size was adjusted by the following formula:

⎥⎦⎤

⎢⎣⎡

−−=

9431'N

ESES

Calculation of the standard error of each estimate of effect size was

accomplished using the following approximation formula:

( )( )21

2

21

21

2 nnES

nnnnse

++

+=

with 2

1se

w = as the weighting factor for ES.

In order to meet the meta-analytical assumption of independence of effect

sizes, only one effect size indicator per study should be represented in an

Initial Report


analysis. If a study used several different outcome measures (for example, some

of the dissertations), then there would be more than one effect size for that study.

In the present meta-analysis, multiple effect sizes per study were averaged in two

ways. If a study used the same subjects with more than one dependent variable

(student achievement tests):

• a weighted mean effect size was calculated whenever the sample sizes

were very divergent (e.g., ni=15~80), and

• an unweighted mean effect size was calculated whenever the sample sizes

were equal or approximately equal.

These averaging procedures resulted in one effect size for each study. Statistical Methods and Analyses

In this study, the software used for calculating the meta-analysis for

standard research design studies was Comprehensive Meta-Analysis® from

BioStat. This software can produce Cohen’s d, Hedges’s g, Q values, confidence

intervals, fixed effects, random effects, and heterogeneity testing results. An

examination of the internal and external validity issues associated with this meta-

analysis was accomplished using multiple linear regression of the moderator

variables. This latter analysis was performed using SPSS® statistical analysis

software.

Initial Report


Results

The results obtained in this investigation are presented in the following

sections: (1) Description of Studies, (2) Meta-Analysis Validity Issues, (3) Meta-

Analysis for All Studies, (4) Meta-Analysis for Studies Classified by Treatment

Categories, and (5) Analysis of Moderator Variables.

Section 1: Description of Studies Characteristics and Frequencies of the Selected Studies

A breakdown of the characteristics of the studies selected for the meta-

analysis is presented in Tables 2 and 3. These characteristics have the potential

to influence the effect sizes obtained from the studies.

Initial Report


Table 2. Frequencies of Variable Characteristics for Included Studies Independent Variable Number of Cases Percent (%)

Publication Year 1980 – 1984 6 9.7 1985 – 1989 7 11.3 1990 – 1994 4 6.5 1995 – 1999 15 24.2 2000 – 2004 30 48.4

Publication Type Refereed Journal Article 40 64.5 Dissertation 18 29.0 Unpublished Report 4 6.5 Type of Study

Experimental (Complete Randomization) 3 4.8

Quasi-Experimental (Randomization Used) 33 53.2

Quasi-Experimental (No Randomization) 26 41.9

Correlational 0 0.0 Test Content Area Biology 17 27.4 Chemistry 12 19.4 Physics 5 8.1 Earth Science 7 11.3 Science 21 33.9 Study Rating Experimental, trt* vs. control 2 3.2 Quasi-, match 1 1.6 Quasi-, similar 1 1.6 Quasi-, single-group pre-post 14 22.6 Quasi-, trt* vs. control pre-post 27 43.5 Quasi-, ANOVA 17 27.4

Totals (for each variable) 62 100.0 *trt = treatment group

Initial Report


The study characteristic Length of Treatment is displayed in Table 3. This

variable was presented separately because the many levels of this attribute result

in numbers of cases that are too small to make any definitive conclusion.

Table 3. Length of Treatment

Length (Months) Number of Cases Percent (%) .10 1 1.6 .25 4 6.5 .50 5 8.1 .75 2 3.2

1.00 8 12.9 2.00 12 19.4 3.00 2 3.2 4.00 3 4.8 5.00 3 4.8 7.50 2 3.2 8.00 1 1.6 9.00 6 9.7

12.00 3 4.8 Missing 10 16.1 Totals 62 100.0

Dependent Variable

The dependent variable (DV) of this meta-analysis, student achievement,

can be referred to by many names (student performance, success, outcomes) and

may be evaluated with a multitude of assessment tools. The types of

assessments used in the studies and their frequencies are presented in Table 4.

Table 4. Dependent Variable (Test Type)

Test Type Number of Cases Percent (%) National Standardized-Multiple Science Content 3 4.8

National Standardized- Single Science Content 6 9.7

Local Standardized- Multiple Science Content 2 3.2

Local Standardized- Single Science Content 4 6.5

Other type test 47 75.8 Totals 62 100.0

Initial Report


In any attempt to synthesize results from different studies, consideration

must be given to the validity and reliability of the different forms of the dependent

variable (DV) that will be used. In this meta-analysis of studies of science

education, 5 different forms of achievement assessment were subjected to

scrutiny.

The most often-encountered form of achievement assessment was local

teacher- or researcher-developed tests. It is assumed that the instruments

constructed by these individuals would possess acceptable levels of logical

relevancy since tests were formulated to match the instructional units that were

being investigated. The degree of reliability of the teacher-made instruments is

unknown as most of the authors did not provide information regarding this

characteristic. Therefore, it was necessary to assume that this form of

assessment (local type test) was conducted with adequate levels of reliability.

For the 15 studies that used standardized national or local assessments of

either multiple or single content areas, an indication of validity was usually not

reported but the authors referred to the original developers of the instruments

when discussing validity. Most authors gave indications of strong content and

construct validity for these forms of assessment. In terms of reliability for these

tests, the range of values reported was .70 to .94. Since these assessment tools

have been widely used and evaluated, the validities and reliabilities are well within

recommended tests and measurement guidelines.

Independent Variables

In order to explore some of the heterogeneity of effect sizes among the

studies, the various treatment conditions were cast into treatment categories.

Rather than operationally define treatment description categories based on the

studies used in this meta-analysis, an established set of teaching strategy

categories (Wise, 1996) was modified and employed. The modified list of

treatment description categories includes the following:

Initial Report


• Questioning strategies. Teachers vary timing, positioning, or cognitive

levels of questions (e.g., increasing wait time, adding pauses at key

student-response points, including more high-cognitive-level questions,

stopping visual media at key points and asking questions, posing

comprehension questions to students at the start of a lesson or

assignment).

• Focusing strategies. Teachers alert students to the intent of the lesson or

capture their attention (e.g., providing objectives or reinforcing objectives at

the middle or closing of lesson, using advance organizers).

• Manipulation strategies. Teachers provide students with opportunities to

work or practice with physical objects (e.g., operating apparatus,

developing skills using manipulatives, drawing or constructing something).

• Enhanced materials strategies. Teachers modify instructional materials

(e.g., rewriting or annotating text materials, tape recording directions,

simplifying laboratory apparatus).

• Testing strategies. Teachers change the frequency, purpose, or cognitive

levels of testing/evaluation (e.g., providing immediate or explanatory

feedback, using diagnostic testing, formative testing, retesting, testing to

mastery).

• Inquiry strategies. Teachers use student-centered, inductive instruction

that is less step-by-step and teacher-directed than traditional instruction

(e.g., using guided or facilitated inquiry activities, guided discoveries,

inductive laboratory exercises, indirect instruction).

• Enhanced context strategies. Teachers relate learning to students’

previous experiences or knowledge or engage students’ interest through

relating learning to the students’/school’s environment or setting (e.g.,

using problem based learning, taking field trips, using the schoolyard for

lessons, encouraging reflection).

• Instructional technology strategies. Teachers use technology to enhance

instruction (e.g., using computers, etc. for simulations, modeling abstract

Initial Report


concepts, and collecting data, showing videos to emphasize a concept,

using pictures, photographs, or diagrams).

• Direct instruction for teaching process skills. Teachers explicitly guide

students through a sequence of tasks (e.g., designing experiments, using

microscope, making measurements).

• Collaborative learning strategies. Teachers arrange students in flexible

groups to work on various tasks (e.g., conducting lab exercises, inquiry

projects, discussions)

A rating sheet (Appendix D) consisting of the treatment categories and a 5-point

Likert-type rating scale was developed to sort each study into an appropriate

category. Three science educators*, each possessing 20-30 years of public

school science instruction experience, read a description of the treatment

condition of each study. They then ranked each of the treatment strategies from 1-

5 for the study. The strategy category receiving the highest ranking response for a

given study was chosen as the treatment category for that study. It should be

noted that while ten strategies are listed, only eight were included in the analysis

because no studies meeting the criteria were found for Focusing strategies or

Direct instruction for teaching process skills. The analysis of the effect sizes for

the different treatment categories is presented in the section concerning

Comprehensive Meta-Analysis® (CMA) outcomes. (Table 8) *These educators were a convenience sample, selected for their expertise in science education and their availability.

Initial Report


Section 2: Meta-Analysis Validity Issues In this meta-analysis, internal validity is concerned with study quality issues

that might influence the effect size obtained for science instruction strategies.

Four variables related to internal validity were examined in this study: (a)

publication type, (b) type of study, (c) study rating, and (d) dependent variable or

achievement test categories.

A reflection of the average effect size, variability and number of studies for

each category of these potential threats to internal validity is presented in Table 5.

Table 5. Internal Validity Influences on the Effect Size of Science Instruction Methodologies Mean SD N Publication Type Refereed Journal Article .91343 .684264 40 Dissertation .28439 .607646 18 Unpublished Report 1.03267 1.194027 3 Type of Study Experimental

(complete randomization) .83467 .375075 3

Quasi-experimental (randomization used) .52571 .677656 33

Quasi-experimental (no randomization) .99608 .775332 25

Study Rating Experimental, trt vs. control .82450 .618718 2 Quasi-, match 1.55100 . 1 Quasi-, similar .51000 . 1 Quasi-, single-group pre-post 1.39831 .827382 13 Quasi-, trt vs. control pre-post .43407 .574647 27 Quasi-, ANOVA .65566 .632238 17 Dependent Variable National Standardized-

Multiple Science Content 1.12367 .722053 3

National Standardized- Single Science Content .58083 .155080 6

Local Standardized- Multiple Science Content .18150 .000707 2

Local Standardized- Single Science Content .92350 .504390 4

Other type test .73568 .807000 46* Totals (for each influence) .73368 .736919 61* *These numbers are 1 less than previously presented due to exclusion of an outlier study.

Initial Report


Because of the diversity in number of studies when these internal validity

concerns are fractionated, it is not prudent to address each level separately

because the number of studies for each will be few. The influences of these

characteristics as a whole are addressed in the section on multiple regression.

External validity deals with generalization issues, such as whether the

effect of science instruction strategies could be generalized to other populations

or situations. In this study, four variables that might influence external validity

were examined: (a) publication year, (b) test content, (c) grade level, and (d)

treatment categories.

A reflection of the average effect size, variability and number of studies for

each category of these potential threats to internal validity is presented in Table 6.

Initial Report


Table 6. External Validity Influences on the Effect Size of Science Instruction Methodology Mean ES Mean SD N Publication Year 1980-1984 .61422 .466598 6 1985-1989 .33086 .163185 7 1990-1994 1.14650 .543101 4 1995-1999 .70471 .884399 14 2000-2004 .81003 .792026 30 Test Content Biology .48466 .755617 17 Chemistry 1.01078 .847695 11 Physics .53680 .258843 5 Earth Science .36643 .504522 7 Science .95940 .719718 21 Grade Level Elementary (K-8)* .65888 .680486 16 High School (9-12) .76027 .761513 45 Treatment Categories Questioning strategies .73913 .529271 3 Manipulation strategies .86213 .825688 8 Enhanced materials

strategies .41336 .480139 12

Testing strategies .49250 .099702 2 Inquiry strategies .62533 .706671 12 Enhanced context

strategies 1.42517 1.050396 6

Instructional media strategies .79160 .774321 15

Collaborative learning strategies .58853 .590639 3

Totals (for each influence) .73368 .736919 61 *When classifying by traditional grade levels, i.e., K-5 as elementary, 6-8 as middle school, the numbers of studies were quite small, therefore these groups were collapsed to the levels presented.

Initial Report


Section 3: Meta-Analysis for All Studies In order to explore the 62 effect sizes obtained in this study, box plots

obtained from SPSS® were constructed. Box plots are used to graphically portray

observations that would be judged to deviate substantially from typical values.

The first box plot is exhibited in Figure 1.

Figure 1. Box Plot of ES for Total Data (N=62)

Effect Size

5

4

3

2

1

0

-1

-2

91

Based on this analysis, study #91 was identified as an extreme outlier,

meaning that its value is substantially different from the median value. The data

for achievement means that were presented in the article were judged to be

suspect by the senior analyst. Therefore, this particular report was excluded from

the analysis and a new box plot was constructed.

Figure 2 illustrates that after removal of the extreme outlier study, two

studies are identified as mild outliers. These studies were not excluded from the

meta-analysis since they were within the ±3 ES range.

Initial Report


Figure 2. Box Plot for Data with Extreme Outlier Removed (N=61)

Effect Size

3

2

1

0

-1

601

38

A pictorial representation of the distribution of effect sizes for this meta-

analysis is shown in Figure 3.

Figure 3. Histogram for the Obtained Effect Sizes (N=61)

3.0002.0001.0000.000-1.000-2.000

Effect Size

25

20

15

10

5

0

Freq

uenc

y

Initial Report


Figure 3 reveals that the largest number of effect sizes occurs in the range

of .00 to .50. With the exception of the indicated most frequent range, the

distribution of effect sizes is relatively normally distributed.

Based on the box plots and histogram, the data after removal of the

extreme outlier was judged to be acceptable for further analysis. A summary of

the analysis of effect sizes for the total data set is presented in Table 7. The

complete output associated with this analysis is given in Appendix F.

Table 7. Meta-Analysis Result for All Studies

N Total Effect Size Lower Limit Upper Limit t p Total

61 Studies 159695 .6696 .6594 .6797 128.9552 .0000

Based on the results across approximately 160,000 observations of

students, there is an effect of science instruction methodology on student

achievement. The effect size of .6696 translates to a treatment performance level

that would be located at the 75 percentile of the control group. This effect size

was judged to be statistically different from .00 as indicated by the probability

value associated with the t statistic.

Initial Report


Section 4: Meta-Analysis for Studies Classified by Treatment Categories

The analyses of the effect sizes for the treatment categories were

effectuated using Comprehensive Meta-Analysis®. A summary of the results of

these analyses is presented in Table 8. For comparison purposes, the results for

the total group are also exhibited in the table. The complete output for the

analyses is exhibited in Appendix F.

Table 8. CMA Results for Total Data and Treatment Categories

N Total Effect Size Lower Limit Upper Limit t-Value Q-Value Total

61 Studies 159695 .6696 .6594 .6797 128.9552 1582.088

Treatment Categories Questioning Strategies 279 .7395 .4927 .9863 5.8992 11.8593

Manipulation Strategies 1240 .5729 .4565 .6893 9.6546 104.8797

Enhanced Material Strategies 2450 .2908 .2100 .3716 7.0540 61.5499

Testing Strategies 166 .5052 .1935 .8169 3.2005 .1906

Inquiry Strategies 145722 .6546 .6440 .6651 121.6432 364.4912

Enhanced Context Strategies 7235 1.4783 1.4167 1.5399 47.0629 147.5931

Instructional Technology Strategies

1962 .4840 .3916 .5764 10.2746 80.1969

Collaborative Learning Strategies 641 .9580 .7773 1.1388 10.4083 26.8652

Several interesting and informative results are presented in Table 8. First,

the effect sizes for all of the treatment categories exceed the lower effect size

benchmark value of .20. Second, two treatment categories, Enhanced Context

Strategies and Collaborative Learning Strategies, exceed the upper effect size

benchmark value of .80. Third, all of these effect sizes would be judged to be

significantly different from zero. The treatment category of Enhanced Material

Strategies exhibited an effect size that would be classified as small.

Initial Report


The 95% confidence intervals for the treatment categories are displayed

graphically in Figure 4 on the following page.

Initi

al R

epor

t

Met

a-A

naly

sis

of N

atio

nal R

esea

rch

Reg

ardi

ng S

cien

ce T

each

ing

24

Figu

re 4

. G

raph

ic R

epre

sent

atio

n of

95%

Con

fiden

ce In

terv

als

for T

reat

men

t Cat

egor

ies

and

Tota

l Dat

a

95 %

Con

fiden

ce In

terv

al

.2

0 .4

0 .6

0 .8

0 1.

00

1.20

1.

40

1.60

1.

80

Que

stio

ning

Str

ateg

ies

Man

ipul

atio

n St

rate

gies

En

hanc

ed M

ater

ial

Stra

tegi

es

Test

ing

Stra

tegi

es

Inqu

iry S

trat

egie

s

En

hanc

ed C

onte

xt

Stra

tegi

es

Inst

ruct

iona

l Tec

hnol

ogy

Stra

tegi

es

Col

labo

rativ

e Le

arni

ng

Stra

tegi

es

Tota

l

Initial Report


Figure 4 clearly shows that the treatment category of Enhanced Context

Strategies is more efficacious than the other strategies. Many of the confidence

intervals for the other treatment strategies contain overlapping ranges.

The central tendency values for the 8 treatment strategies are displayed in bar

chart form in Figure 5.

Figure 5. Mean Effect Sizes for Treatment Categories and Total Data

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

C1* C2* C3* C4* C5* C6* C7* C8* TotalMean ES

*Note: C1=Questioning Strategies

C2=Manipulation Strategies C3=Enhanced Material Strategies C4=Testing Strategies C5=Inquiry Strategies C6=Enhanced Context Strategies C7=Instructional Technology Strategies C8=Collaborative Learning Strategies

When making decisions based on the results of meta-analysis, one is always

concerned with what is known as the “file drawer problem.” This problem refers to the

idea that only studies that result in an effect are included in a meta-analysis. Studies

that did not uncover an effect are usually filed away by the researchers and are not

Initial Report


available for inclusion. A calculation that attempts to address this concern is the

Failsafe N (Nfs). The obtained numerical value from this statistic is an estimate of the

number of non-significant “file drawer” studies that would need to be obtained and

included in a meta-analysis before a statement of no effect would be given to the

completed meta-analysis. Failsafe Ns for the total data set and the studies in the

treatment categories are exhibited in Table 9.

Table 9. Failsafe N for Total Data and Treatment Description Categories

Data ES N Nfs Overall .6696 61 756 Questioning Strategies .7395 3 42 Manipulation Strategies .5729 8 84 Enhanced Material Strategies .2908 12 58 Testing Strategies .5052 2 19 Inquiry Strategies .6546 12 145 Enhanced Context Strategies 1.4783 6 172 Instructional Technology Strategies .4840 15 130 Collaborative Learning Strategies .9580 3 55

Based on the estimates presented in Table 9, the only questionable treatment

category effect size in terms of a file drawer problem is for Testing Strategies. If 19 non-

significant results were found in file drawers, a decision of no effect for this strategy

would be stated. The resultant Nfs for the overall data would lead one to conclude that a

decision to deny an effect is not probable given that approximately 756 non-significant

studies would be needed to reverse the decision about the overall effect.

Initial Report


Section 5: Analysis of Moderator Variables The moderator variables were placed in a multiple regression analysis to

determine the influence of all the coding components on the effect size data. A

summary of the SPSS® results is presented in Tables 10 and 11 and the complete

printout is exhibited in Appendix G.

Table 10. Regression Analysis for Moderator Variables Source Sum of Squares df Mean Square F Sig. Regression 4.243 9 .471 1.102 .383 Residual 17.536 41 .428 Total 21.778 50

Table 11. Regression Analysis: Dependent Variable: Effect Size (N=61) Moderator Variable Beta t Sig. Type of study .151 1.052 .297 Grade Level .067 .474 .638 Treatment Categories .036 .254 .801 Publish Type -.233 -1.591 .118 Study Rating -.181 -1.279 .207 Test Content .064 .438 .663 Dependent Variable .066 .488 .628 Publication Year .117 .696 .489 Length of Treatment .342 1.532 .133 Note. R2 = .167. The effect size outlier had been excluded in this analysis.

From the results of the regression analysis, it can be seen that the F ratio

associated with this data indicates that there is not a relationship between the

moderator variables and the dependent variable of effect size. The beta weights for the

regression were all judged to be non-significant. This means that the effect sizes

obtained in this meta-analysis are not influenced by the fact that the studies have

different levels of these potentially contaminating attributes.

Initial Report


Conclusions

What teaching methodologies have been shown to improve student achievement in science? Conclusions from this meta-analysis can be framed based

on the rankings of the effect sizes associated with the respective teaching strategies. A

ranking of the strategies is presented in Table 12.

Table 12. Ranking of Teaching Strategies Strategies Effect Size Rank

Enhanced Context Strategies 1.4783 1

Collaborative Learning Strategies .9580 2

Questioning Strategies .7395 3

Inquiry Strategies .6546 4

Manipulation Strategies .5729 5

Testing Strategies .5052 6

Instructional Technology Strategies .4840 7

Enhanced Material Strategies .2908 8

All of the innovative teaching strategies presented in the 61 studies exhibited a

positive influence on student achievement. As indicated by Wise (1996), innovative

science instruction is a mixture of teaching strategies and no one strategy is as powerful

as utilizing a combined strategies approach. Students exposed to a traditional approach

of science instruction can and will exhibit achievement; however, meta-analysis results

indicate that incorporation of other avenues of learning via innovative strategies

significantly augments the degree of achievement.

Within the family of instructional strategies, Enhanced Context Strategies such as

relating to previous learning, field trips, group discussion, games, simulations, and

reflective learning seem to have the greatest impact. Collaborative Learning Strategies

such as flexible heterogeneous groupings and interdisciplinary teaming also displayed a

strong effect. The results are not influenced by the variety of study characteristics and

settings including grade level of students and subject area within science.

Initial Report


The teaching strategy category that exhibited the largest effect size was

Enhanced Context Strategies. Teachers make learning relevant to students by

presenting material in the context of real-world examples and problems. The real world

can be brought to students through technology and students may be taken out of the

classroom into the real world through field experiences. This type of augmented

instruction is aligned with the implications for teaching outlined in How People Learn:

Brain Mind, Experience and School (Bransford et al. 2000):

1. Teachers must draw out and work with the preexisting understandings that their

students bring with them. (p. 19)

2. Teachers must teach some subject matter in depth, providing many examples in

which the same concept is at work and providing a firm foundation of factual

knowledge. (p. 20)

3. The teaching of metacognitive skills should be integrated into the curriculum in a

variety of subject areas. (p. 21) Students should be encouraged to reflect on their

learning through journaling and self-assessment activities.

When considering the meta-analysis of the 61 studies as a whole, one is forced to

conclude that when science instruction is altered from the traditional or control

approach, student achievement is enhanced. While this result is very important, it is not

unexpected. If students are placed in an environment in which they can (1) actively

connect the instruction to their interests and present understandings, (2) experience

success early in the learning process, and (3) have an opportunity to experience

scientific inquiry, achievement will be accelerated.

Initial Report


References Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain,

mind, experience, and school (Expanded ed.). Washington, D.C.: National Academy Press.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York: Academic Press.

Wise, K. C. (1996). Strategies for teaching science: What works? Clearing House, 69(6), 337-338.

Initial Report


Studies Included in Meta-analysis* Akpan, J. P. & Andre, T. (2000). Using a computer simulation before dissection to help students learn

anatomy. Journal of Computers in Mathematics and Science Teaching, 19(3), 297-313. (475) Baker, T. R., & White, S. H. (2003). The effects of GIS on students’ attitudes, self-efficacy, and

achievement in middle school science classrooms. Journal of Geography, 102(6), 243-254. (48)

Berube, C. T. (2001). A study of the effects of constructivist-based vs. traditional direct instruction on 8th grade science comprehension. Unpublished Ph.D., Old Dominion University, United States – Virginia. (21)

Bluette, C. L., Jr. (1999). An evaluation of the effects of staff development for teachers who utilize multiple strategies with middle school students in mathematics and science. Unpublished Ed.D., Saint Louis University, United States – Missouri. (32)

Chang, H. P., & Lederman, N. G. (1994). The Effect Of Levels Of Cooperation Within Physical Science Laboratory Groups On Physical Science Achievement. Journal Of Research In Science Teaching, 31(2), 167-181. (166)

Clark, D. R. (2000). Effects of teaching high school chemistry with dynamic particle models on student achievement and conceptual understanding. Unpublished Ph.D., The Catholic University of America, United States – District of Columbia. (30, 300)

Dalton, B., Morocco, C. C., Tivnan, T., & Mead, P. L. R. (1997). Supported inquiry science: Teaching for conceptual change in urban and suburban science classrooms. Journal Of Learning Disabilities, 30(6), 670-684. (448)

Dean, D. M. (2004). An evaluation of the use of Web-enhanced homework assignments in high school biology classes. Unpublished Ed.D., The University of Alabama, United States – Alabama. (1)

Demirci, N. (2001). The effects of a Web-based physics software program on students’ achievement and misconceptions in force and motion concepts. Unpublished Ph.D., Florida Institute of Technology, United States – Florida. (25)

Dillashaw, F. G., & Okey, J. R. (1983). Effects of a modified mastery learning strategy on achievement, attitudes, and on-task behavior of high school chemistry students. Journal of Research In Science Teaching, 20(3), 203-211. (457)

Faro, S. T. (2003). An investigation of components of the studio model and supplemental online materials, on student achievement and attitudes in science at the high school level. Unpublished Ph.D., State University of New York at Albany, United States – New York. (4)

Fortus, D., Dershimer, R. C., Krajcik, J., Marx, R. W., & Mamlok-Naaman, R. (2004). Design-based science and student learning. Journal of Research in Science Teaching, 41(10), 1081-1110. (60, 600, 601)

Frederick, L. R., & Shaw, E. L., Jr. (1999, November 17-19). Effects of science manipulatives on achievement, attitudes, and journal writing of elementary science students revisited. Paper presented at the Annual Meeting of the Mid-South Educational Research Association, Point Clear, AL. (ERIC Document Reproduction Service No. ED 436410). (468)

Frederick, L. R., & Shaw, E. L., Jr. (1998, November 4-6). Examining the effects of science manipulatives

on achievement, attitudes, and journal writing of elementary science students. Paper presented at

Initial Report


the Annual Meeting of the Mid-South Educational Research Association, New Orleans, LA. (ERIC Document Reproduction Service No. ED 427975). (91)

Gatlin, L. S. (1998). The effect of pedagogy informed by constructivism: A comparison of student

achievement across constructivist and traditional classroom environments. Unpublished Ph.D., University of New Orleans, United States – Louisiana. (38)

Glasson, G. E. (1989). The effects of hands-on and teacher demonstration laboratory methods on science achievement in relation to reasoning ability and prior knowledge. Journal of Research In Science Teaching, 26(2), 121-131. (276)

Hamrick, L., & Harty, H. (1987). Influence of resequencing general science content on the science achievement, attitudes toward science, and interest in science of 6th grade students. Journal of Research In Science Teaching, 24(1), 15-25. (278)

Hand, B., Wallace, C. W., & Yang, E. M. (2004). Using a science writing heuristic to enhance learning outcomes from laboratory activities in seventh-grade science: Quantitative and qualitative aspects. International Journal of Science Education, 26(2), 131-149. (50)

Harwood, W. S., & McMahon, M. M. (1997). Effects of integrated video media on student achievement and attitudes in high school chemistry. Journal of Research in Science Teaching, 34(6), 617-631. (57)

Hocutt, M. M. (2003). Comparing instructional methodologies in sixth-grade science: Traditional textbook, integrated science, and integrated science with technology enhancement. Unpublished Ph.D., The University of Alabama, United States – Alabama. (6, 609)

Houtz, L. K. E. (1995). Instructional strategy change and the attitude and achievement of 7th-grade and 8th-grade science students. Journal of Research In Science Teaching, 32(6), 629-648. (245, 2450)

Huffman, D., Goldberg, F., & Michlin, M. (2003). Using computers to create constructivist learning environments: Impact on pedagogy and achievement. Journal of Computers in Mathematics and Science Teaching, 22(2), 151-168. (452, 4520)

Jafer, Y. J. (2003). The effects of computer-assisted instruction on fourth-grade students’ achievement and attitudes toward desert issues. Unpublished Ph.D., Utah State University, United States – Utah. (7)

Johnson, R. T., Johnson, D. W., Scott, L. E., & Ramolae, B. A. (1985). Effects of single-sex and mixed-sex cooperative interaction on science achievement and attitudes and cross-handicap and cross-sex relationships. Journal of Research In Science Teaching, 22(3), 207-220. (285)

Kariuki, P., & Paulson, R. (2001, November 14-17). The effects of computer animated dissection versus preserved animal dissection on the student achievement in a high school biology class. Paper presented at the Annual Conference of the Mid-South Educational Research Association, Little Rock, AK. (ERIC Document Reproduction Service No. ED460018). (467)

Kim, N. B. (1997). A comparison of the effects of computer-enhanced with traditional instruction on the

learning outcomes of high-school students in anatomy classes. Unpublished Ph.D., University of Pittsburgh, United States – Pennsylvania. (39)

Kremer, P. L. (1983). Effects of individualized assignments on biology achievement. Journal of Research In Science Teaching, 20(2), 105-115. (170)

Initial Report


Lavoie, D. R. (1999). Effects of emphasizing hypothetico-predictive reasoning within the science learning cycle on high school student’s process skills and conceptual understandings in biology. Journal of Research in Science Teaching, 36(10), 1127-1147. (64)

Long, J. C., Okey, J. R., & Yeany, R. H. (1981). The effects of a diagnostic-prescriptive teaching strategy on student achievement in biology. Journal of Research In Science Teaching, 18(6), 515-523. (455, 456)

Louden, C. K. (1997). Teaching strategies and student achievement in high school block scheduled biology classes. Unpublished Ph.D., The University of North Carolina at Chapel Hill, United States – North Carolina. (41)

Matthews, D. R., & McLaughlin, T. F. (1994). Effects of learner-centered laboratory activities on achievement and students preferences in 2 high-school biology courses. Perceptual and Motor Skills, 78(1), 285-286. (247)

Marszalek, C. S. (1998). Effects on seventh-grade students’ achievement and science anxiety of alternatives to conventional frog dissection. Unpublished Ed.D., Northern Illinois University, United States – Illinois. (47, 470)

Marx, R. W., Blumenfeld, P. C., Krajcik, J. S., Fishman, B., Soloway, E., Geier, R., et al. (2004). Inquiry-based science in the middle grades: Assessment of learning in urban systemic reform. Journal of Research in Science Teaching, 41(10), 1063-1080. (78, 780, 781, 782)

McManus, D. O., Dunn, R., & Denig, S. (2003). Effects of traditional lecture versus teacher-constructed & student-constructed self-teaching instructional resources on short-term science achievement & attitudes. American Biology Teacher, 65(2), 93-102. (229)

Orehowsky, W. (1999). The effect of laboratory-based instruction and assessment on student attitudes toward the laboratory experience and achievement in chemistry at the high school level. Unpublished Ed.D., Temple University, United States – Pennsylvania. (36)

Osman, M. E., & Hannafin, M. J. (1994). Effects of advance questioning and prior knowledge on science learning. Journal of Educational Research, 88(1), 5-13. (70)

Pallant, A., & Tinker, R. (2004). Reasoning with atomic-scale molecular dynamic models. Journal of Science Education and Technology, 13(1), 51. (75, 750)

Purser, R. K., & Renner, J. W. (1983). Results of two tenth-grade biology teaching procedures. Science Education, 67(1), 85-98. (433)

Riley, J. P. (1986). The effects of teachers wait-time and knowledge comprehension questioning on science achievement. Journal of Research In Science Teaching, 23(4), 335-342. (281, 2810)

Roberts, P. H. (1999). Effects of multisensory resources on the achievement and science attitudes of seventh-grade suburban students taught science concepts on and above grade level. Unpublished Ed.D., St. John’s University (New York), United States – New York. (33)

Romance, N. R., & Vitale, M. R. (1992). A curriculum strategy that expands time for in-depth elementary science instruction by using science-based reading strategies: Effects of a year-long study in grade four. Journal of Research in Science Teaching, 29(6), 545-554. (439)

Rose-Baele, J. S. (2003). Report of Fifth Grade Outcome Study, Science For All Students, 2001-2002. Retrieved June 6, 2005, from http://www.ccsdschools.com/administration/assessment/PIpage.html (150)

Initial Report


Saunders, W. L., & Shepardson, D. (1987). A comparison of concrete and formal science instruction upon science achievement and reasoning ability of 6th grade students. Journal of Research in Science Teaching, 24(1), 39-51. (90)

Schneider, L. S., & Renner, J. W. (1980). Concrete and formal teaching. Journal of Research in Science Teaching, 17(6), 503-517. (434)

Sherris, J. D., & Kahle, J. B. (1984). The effects of instructional organization and locus of control orientation on meaningful learning in high school biology students. Journal of Research In Science Teaching, 21(1), 83-94. (451)

Songer, N. B., Lee, H. S., & McDonald, S. (2003). Research towards an expanded understanding of inquiry science beyond one idealized standard. Science Education, 87(4), 490-516. (142, 1420)

Turpin, T. J. (2000). A study of the effects of an integrated, activity-based science curriculum on student achievement, science process skills, and science attitudes. Unpublished Ed.D., University of Louisiana at Monroe, United States – Louisiana. (29)

Yager, R. E. (1989). Comparison of standard student performance when science study is organized around typical concepts. Bulletin of Science, Technology, & Society, 9, 171-181. (98)

Yager, R. E., & Weld, J. D. (1999). Scope, sequence and coordination: The Iowa Project, a national reform effort in the USA. International Journal of Science Education, 21(2), 169-194. (369)

* Number in parentheses at end of citation is study number.

Initial Report


Appendix A Web Sites and Search Terms

Initial Report


Web Sites

Berkeley National Laboratory (http://www.lbl.gov/Science-Articles/) Department of Education (http://www.ed.gov/index.jhtml) National Academies (http://www.nas.edu/) National Science Resources (http://www.nsrconline.org/) American Association for the Advancement of Science (AAAS)

(http://www.aaas.org/programs/education/) Education Development Center (http://main.edc.org/) National Science Teachers Association (http://www.nsta.org/resources) Department of Education Expert Panel on Mathematics and Science Education

(http://www.ed.gov/offices/OERI/ORAD/KAD/expert_panel/newscience_progs.html)

Search Terms

authentic science & achievement authentic science & student achievement authentic science & student outcomes authentic science & student performance biology & achievement biology & student achievement biology & student outcomes biology & student performance chemistry & achievement chemistry & student achievement chemistry & student outcomes chemistry & student performance constructivis* & science & student outcomes constructivis* & science & teaching constructivis* & science & teaching & achievement constructivist & science constructivist & science & student achievement constructivist science & achievement constructivist science & student achievement constructivist science & student outcomes constructivist science & student progress discovery learning & achievement discovery learning & student achievement earth & student achievement earth & student achievement & science earth & student outcomes earth & student outcomes & science earth & student performance earth & student performance & science earth science & achievement ecology & student achievement ecology & student outcomes ecology & student outcomes & science ecology & student performance elementary school science & achievement environmental education and student*

Initial Report


environmental science and student performance environmental science and student performance hands-on science & achievement hands-on science & outcome* hands-on science & performance hands-on science & student achievement hands-on science & student outcome* hands-on science & student performance middle school science & achievement physic* & student achievement physic* & student achievement & science physic* & student outcomes physic* & student outcomes & science physic* & student performance physic* & student performance & science physic* science & achievement physics & achievement professional development & achievement & science professional development & science professional development & science teaching professional development & student achievement & science science & achievement science & class* performance science & curriculum assessment science & education & treatment & control & (achievement or outcome* or success) science & student achievement science & student outcome science & student performance science & student progress science & student success science education & achievement science education & curriculum assessment science education & progress science education & student achievement science education & student outcome science education & student performance science education & student progress science education & student success science learning & achievement science learning & student achievement science learning & student outcome teaching methods & science & achievement teaching methods & science achievement

Initial Report


Appendix B Bibliography of Studies and Articles

Initial Report


Abraham, M. R., & Renner, J. W. (1986). The sequence of learning cycle activities in high school

chemistry. Journal of Research In Science Teaching, 23(2), 121-143.

Adamson, S. L., Banks, D., Burtch, M., Cox, F., Judson, E., Turley, J. B., et al. (2003). Reformed undergraduate instruction and its subsequent impact on secondary school teaching practice and student achievement. Journal of Research In Science Teaching, 40(10), 939-957.

Akpan, J. P., & Andre, T. (1999). The effect of a prior dissection simulation on middle school students' dissection performance and understanding of the anatomy and morphology of the frog. Journal of Science Education and Technology, 8(2), 107-121.

Akpan, J. P., & Andre, T. (2000). Using a computer simulation before dissection to help students learn anatomy. Journal of Computers in Mathematics and Science Teaching, 19(3), 297-313.

Aleven, V. A. W. M. M., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2), 147-179.

Alparslan, C., Tekkaya, C., & Geban, O. (2003). Using the conceptual change instruction to improve learning. Journal of Biological Education, 37(3), 133-137.

Amaral, O. M., Garrison, L., & Klentschy, M. (2002). Helping English learners increase achievement through inquiry-based science instruction. Bilingual Research Journal, 26(2), 213-239.

Anderson, R. D. (1983). A consolidation and appraisal of science meta-analyses. Journal of Research In Science Teaching, 20(5), 497-509.

Anderson, R. D., Kahl, S. R., Glass, G. V., & Smith, M. L. (1983). Science education: A meta-analysis of major questions. Journal of Research In Science Teaching, 20(5), 379-385.

Andre, T., & Ding, P. (1991). Student misconceptions, declarative knowledge, stimulus conditions, and problem solving in basic electricity. Contemporary Educational Psychology, 16(4), 303-313.

Baker, D. R. (1991a). Process skills acquisition, cognitive growth, and attitude change of ninth grade students in a scientific literacy course. Journal of Research In Science Teaching, 28(5), 423-436.

Baker, D. R. (1991b). A summary of research in science-education - 1989. Science Education, 75(3), R5-402.

Baker, T. R. (2002). The effects of geographic information system (GIS) technologies on students' attitudes, self-efficacy, and achievement in middle school science classrooms. Unpublished Ph.D., University of Kansas, United States -- Kansas.

Baker, T. R., & White, S. H. (2003). The effects of GIS on students' attitudes, self-efficacy, and achievement in middle school science classrooms. Journal of Geography, 102(6), 243-254.

Bandlow, R. J. (2001). The misdirection of middle school reform. Clearing House, 75(2), 69-73.

Banilower, E. R. (2002). Results of the 2001-2002 study of the impact of the Local Systemic Change Initiative on student achievement in science. Retrieved May 19, 2005, from http://www.horizon-research.com/reports/2002/sps0102.php

Barnett, M. (2005). Engaging inner city students in learning through designing remote operated vehicles. Journal of Science Education and Technology, 14(1), 87.

Initial Report


Barron, B. J. S., Schwartz, D., Vye, N. J., Moore, A., Petrosino, A. J., Zech, L., et al. (1998). Doing with understanding: Lessons from research on problem- and project-based learning. The Journal of the Learning Sciences, 7(3&4), 271-311.

Bay, M. (1992). Science instruction for the mildly handicapped: Direct instruction versus discovery teaching. Journal of Research In Science Teaching, 29(6), 555-570.

Bayraktar, S. (2000). A meta-analysis on the effectiveness of computer-assisted instruction in science education. Unpublished Ph.D., Ohio University, United States -- Ohio.

Beasley, W. (1985). Improving student laboratory performance: How much practice makes perfect. Science Education, 69(4), 567-576.

Beeth, M. E. (1998). Teaching science in fifth grade: Instructional goals that support conceptual change. Journal of Research in Science Teaching, 35(10), 1091-1101.

Bell, R., & Bell, L. (2003). A bibliography of articles on technology in science education. Contemporary Issues in Technology and Teacher Education, 2(4), 427-447.

Benson, J. S., & Yeany, R. H. (1980, April). Generalizability of diagnostic-prescriptive teaching strategies across student locus of control and multiple instructional units. Paper presented at the Annual Meeting of the American Educational Research Association, Boston, MA. (ERIC Document Reproduction Service No. ED187534).

Berube, C. T. (2001). A study of the effects of constructivist-based vs. traditional direct instruction on 8th

grade science comprehension. Unpublished Ph.D., Old Dominion University, United States -- Virginia.

Bianchini, J. A. (1999). From here to equity: The influence of status on student access to and understanding of science. Science Education, 83(5), 577-601.

Blough, R. (1996). The effects of using student response keypads on student motivation and achievement in high school biology classes: A time series design. Unpublished Ed.D., The University of Tennessee, United States -- Tennessee.

Bluette, C. L., Jr. (1999). An evaluation of the effects of staff development for teachers who utilize multiple strategies with middle school students in mathematics and science. Unpublished Ed.D., Saint Louis University, United States -- Missouri.

Blumenfeld, P. C., & Meece, J. L. (1988). Task factors, teacher behavior, and students' involvement and use of learning strategies in science. The Elementary School Journal, 88(3), 235-250.

Boddy, N., Watson, K., & Aubusson, P. (2003). A trial of the Five Es: A referent model for constructivist teaching and learning. Research In Science Education, 33(1), 27-42.

Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2002). Comprehensive school reform and student achievement: A meta-analysis. Baltimore, MD: Center for Research on the Education of Students Placed At Risk (CRESPAR), The Johns Hopkins University.

BouJaoude, S. (1998, April 19-22). Analogies, summaries, and question answering in middle school life science: Effect on achievement and perceptions of instructional value. Paper presented at the Annual Conference of the National Association for Research in Science Teaching, San Diego, CA. (ERIC Document Reproduction Service No. ED420503).

Initial Report


BouJaoude, S., & Attich, M. (2003, March 23-26). The effect of using concept maps as study tools on achievement in chemistry. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, Philadelphia, PA. (ERIC Document Reproduction Service No. ED477305).

Bowen, C. W. (2000). A quantitative literature review of cooperative learning effects on high school and

college chemistry achievement. Journal Of Chemical Education, 77(1), 116-119.

Bredderman, T. (1983). Effects of activity-based elementary science on student outcomes - A quantitative synthesis. Review of Educational Research, 53(4), 499-518.

Brookhart, S. M. (1997). Effects of the classroom assessment environment on mathematics and science achievement. Journal Of Educational Research, 90(6), 323-330.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.

Brunkhorst, B. J. (1987). Exemplary middle and junior-high science programs - Student outcomes in knowledge, attitudes and STS applications. Bulletin of Science Technology & Society, 7(5-6), 813-824.

Brunkhorst, B. J. (1992). A study of student outcomes and teacher characteristics in exemplary middle and junior-high science programs. Journal of Research in Science Teaching, 29(6), 571-583.

Bunce, D. M., & Heikkinen, H. (1986). The Effects Of An Explicit Problem-Solving Approach On Mathematical Chemistry Achievement. Journal Of Research In Science Teaching, 23(1), 11-20.

Burkam, D. T., Lee, V. E., & Smerdon, B. A. (1997). Gender and science learning early in high school: Subject matter and laboratory experiences. American Educational Research Journal, 34(2), 297-331.

Calyk, M., Ayas, A., & Ebenezer, J. V. (2005). A review of solution chemistry studies: Insights into students' conceptions. Journal of Science Education and Technology, 14(1), 29-50.

Carey, S. (2000). Science education as conceptual change. Journal of Applied Developmental Psychology, 21(1), 13-19.

Carey, S., Evans, R., Honda, M., Jay, E., & Unger, C. (1989). An experiment is when you try it and see if it works - A study of grade-7 students understanding of the construction of scientific knowledge. International Journal of Science Education, 11(5), 514-529.

Carlsen, D. D., & Andre, T. (1992). Use of a microcomputer simulation and conceptual change text to overcome student preconceptions about electric circuits. Journal of Computer-Based Instruction, 19(4), 105-109.

Carpenter, T. P., Blanton, M. L., Cobb, P., Franke, M. L., Kaput, J., & McClain, K. (2004). Scaling up innovative practices in mathematics and science. Retrieved May 27, 2005, from http://ia108025.us.archive.org/1/items/Scaling_Up_Innovative_Practices_in_Mathematics_and_Science/NCISLAReport1.pdf

Chambers, S. K., & Andre, T. (1997). Gender, prior knowledge, interest, and experience in electricity and conceptual change text manipulations in learning about direct current. Journal of Research in Science Teaching, 34(2), 107-123.

Initial Report


Chang, C. Y. (2001). Comparing the impacts of a problem-based computer-assisted instruction and the direct-interactive teaching method on student science achievement. Journal of Science Education and Technology, 10(2), 147-153.

Chang, C. Y. (2002a). Does computer-assisted instruction + problem solving = improved science outcomes? A pioneer study. Journal Of Educational Research, 95(3), 143-150.

Chang, C. Y. (2002b). The impact of different forms of multimedia CAI on students' science achievement. Innovations In Education And Teaching International, 39(4), 280-288.

Chang, C. Y. (2003). Teaching earth sciences: should we implement teacher-directed or student-controlled CAI in the secondary classroom? International Journal Of Science Education, 25(4), 427-438.

Chang, C. Y., & Barufaldi, J. P. (1999). The use of a problem-solving-based instructional model in initiating change in students' achievement and alternative frameworks. International Journal of Science Education, 21(4), 373-388.

Chang, C. Y., & Mao, S. L. (1999). Comparison of Taiwan science students' outcomes with inquiry-group versus traditional instruction. Journal Of Educational Research, 92(6), 340-346.

Chang, H. P., & Lederman, N. G. (1994). The effect of levels of cooperation within physical science laboratory groups on physical science achievement. Journal of Research in Science Teaching, 31(2), 167-181.

Chao, C. I., Ruiz, L., & Reigeluth, C. M. (1983). Effects of four instructional sequences on application and transfer. IDD&E Working Paper No. 12. School of Education, Syracuse University. (ERIC Document Reproduction Service No. ED289461).

Chen, Z., & Klahr, D. (1999). All other things being equal: Acquisition and transfer of the control of

variables strategy. Child Development, 70(5), 1098-1120.

Christman, J. B. (2001). Powerful ideas, modest gains: Five years of systemic reform in Philadelphia middle schools. Retrieved May 23, 2005, from http://www.cpre.org/Publications/children05.pdf

Clark, D. R. (2000). Effects of teaching high school chemistry with dynamic particle models on student achievement and conceptual understanding. Unpublished Ph.D., The Catholic University of America, United States -- District of Columbia.

Clavijo, K. G. (2002). The impact of environmental education on sixth-grade students' science achievement. Unpublished Ed.D., University of Louisville, United States -- Kentucky.

Clewell, B. C., Cosentino de Cohen, C., Deterding, N., Manes, S., & Tsui, L. (2005). What do we know? Seeking effective math and science instruction. Retrieved May 20, 2005, from http://www.urban.org/UploadedPDF/311150.pdf

Clewell, B. C., de Cohen, C. C., Campbell, P. B., & Perlman, L. (2004). Review of evaluation studies of mathematics and science curricula and professional development models. Retrieved May 24, 2005, from http://www.urban.org/Template.cfm?NavMenuID=24&template=/TaggedContent/ViewPublication.cfm&PublicationID=9195

Clune, W. (1998). Toward a theory of systemic reform: The case of nine NSF statewide systemic initiatives. Retrieved May 23, 2005, from http://www.wcer.wisc.edu/archive/nise/Publications/Research_Monographs/vol16.pdf

Initial Report


Cohen, H. G. (1982). Relationship between locus of control and the development of spatial conceptual abilities. Science Education, 66(4), 635-642.

Coleman, E. B. (1998). Using explanatory knowledge during collaborative problem solving in science. Journal of the Learning Sciences, 7(3&4), 387-427.

Comeaux, P., & Huber, R. (2001). Students as scientists: Using interactive technologies and collaborative inquiry in an environmental science project for teachers and their students. Journal of Science Teacher Education, 12(4), 235.

Concoran, T. B., Shields, P. M., & Zucker, A. A. (1998). The SSIs and professional development for teachers. Retrieved May 23, 2005

Cozzens, M. B. (1996). Instructional materials development (IMD): A review of the IMD program, past, present, and future. Retrieved May 23, 2005, from http://teech.terc.edu/papers/papers/cozzens.htm

Crow, L. W., & Haws, S. G. (1985, April). The effects of teaching logical reasoning upon students' critical thinking and science achievement. Paper presented at the 58th Annual Meeting of the National Association for Research in Science Teaching, Indiana. (ERIC Document Reproduction Service No. ED255371).

Czemiak, C. M., Beltyukova, S., Struble, J., Haney, J. J., & Lumpe, A. T. (2005). Do you see what I see?

The relationship between a professional development model and student achievement. In R. E. Yager (Ed.), Exemplary science in grades 5-8: Standards-based success stories. Arlington, VA: NSTA Press.

Dalton, B., Morocco, C. C., Tivnan, T., & Mead, P. L. R. (1997). Supported inquiry science: Teaching for conceptual change in urban and suburban science classrooms. Journal Of Learning Disabilities, 30(6), 670-684.

Dana Center, & University of Texas at Austin. (1999). Hope for urban education: A study of nine high-performing, high-poverty, urban elementary schools. Retrieved May 24, 2005, from http://www.ed.gov/PDFDocs/urbaned.pdf

Darling-Hammond, L., Berry, B., & Thoreson, A. (2000). Does teacher certification matter? Evaluating the evidence. Retrieved June 7, 2005, from http://www.teachingquality.org/resources/pdfs/doescertificationmatter.pdf

De Baz, T. (2001). The effectiveness of;the jugsaw cooperative learning on students' achievement and attitudes toward science. Science Education International, 12(4), 6-11.

Dean, D. M. (2004). An evaluation of the use of Web-enhanced homework assignments in high school biology classes. Unpublished Ed.D., The University of Alabama, United States -- Alabama.

Demirci, N. (2001). The effects of a Web-based physics software program on students' achievement and misconceptions in force and motion concepts. Unpublished Ph.D., Florida Institute of Technology, United States -- Florida.

Desimone, L., Garet, M. S., Birman, B. F., Porter, A., & Yoon, K. S. (2003). Improving teachers' in-service professional development in mathematics and science: The role of postsecondary institutions. Educational Policy, 17(5), 613-649.

Initial Report


Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers' instruction: Results from a three-year longitudinal study. Educational Evaluation And Policy Analysis, 24(2), 81-112.

Dethlefs, T. M. (2002). Relationship of constructivist learning environment to student attitudes and achievement in high school mathematics and science. Unpublished Ph.D., The University of Nebraska - Lincoln, United States -- Nebraska.

Dickson, T. K. (2002). Assessing the effect of inquiry-based professional development on science achievement tests scores. Unpublished Ed.D., University of North Texas, United States -- Texas.

Dillashaw, F. G., & Okey, J. R. (1983). Effects of a modified mastery learning strategy on achievement, attitudes, and on-task behavior of high school chemistry students. Journal of Research In Science Teaching, 20(3), 203-211.

Dimitrov, D. M., McGee, S., & Howard, B. C. (2002). Changes in students' science ability produced by multimedia learning environments: application of the linear logistic model for change. School Science and Mathematics, 102(1), 15-24.

Donovan, M. S., & Bransford, J. D. (Eds.). (2005). How students learn: History, mathematics, and science in the classroom. Washington, D.C.: The National Academies Press.

Dori, Y. J., & Hameiri, M. (2003). Multidimensional analysis system for quantitative chemistry problems: Symbol, macro, micro, and process aspects. Journal Of Research In Science Teaching, 40(3), 278-302.

Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young people's images of science. Philadelphia: Open University Press.

Druva, C. A., & Anderson, R. D. (1983). Science teacher characteristics by teacher-behavior and by student outcome - A meta-analysis of research. Journal of Research in Science Teaching, 20(5), 467-479.

Dunbar, K. (2000). How scientists think in the real world: Implications for science education. Journal of Applied Developmental Psychology, 21(1), 49-58.

Dykstra, D. I. (2004a). Why teach kinematics? An examination of the teaching of kinematics and force - Part I. Manuscript submitted for publication.

Dykstra, D. I. (2004b). Why teach kinematics? An examination of the teaching of kinematics and force - Part II. Manuscript submitted for publication.

Ertepinar, H., & Geban, O. (1996). Effect of instruction supplied with the investigative-oriented laboratory approach on achievement in a science course. Educational Research, 38(3), 333-341.

Eryilmaz, A. (1996). The effects of conceptual assignments, conceptual change discussions, and a CAI program emphasizing cognitive conflict on students' achievement and misconceptions in physics. Unpublished Ph.D., Florida Institute of Technology, United States -- Florida.

Faro, S. T. (2003). An investigation of components of the studio model and supplemental online materials, on student achievement and attitudes in science at the high school level. Unpublished Ph.D., State University of New York at Albany, United States -- New York.

Fenster, M. J. (1998, April 13-17). Evaluating the impact of science, math and technology initiatives on student achievement: The case of the New Jersey Statewide Systemic Initiative (NJSSI). Paper

Initial Report


presented at the Annual Meeting of the American Educational Research Association, San Diego, CA. (ERIC Document Reproduction Service No. ED424292).

Ferrara, S., Huynh, H., & Michaels, H. (1999). Contextual explanations of local dependence in item

clusters in a large scale hands-on science performance assessment. Journal of Educational Measurement, 36(2), 119-140.

Fisher, D., Henderson, D., & Fraser, B. (1997). Laboratory environments & student outcomes in senior high school biology. The American Biology Teacher, 59(4), 214-219.

Fishman, B. J., Marx, R. W., Best, S., & Tal, R. T. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education, 19(6), 643-658.

Fortus, D., Dershimer, R. C., Krajcik, J., Marx, R. W., & Mamlok-Naaman, R. (2004). Design-based science and student learning. Journal of Research in Science Teaching, 41(10), 1081-1110.

Foster, J. C. (1997). The relationships between integration strategies and student achievement scores in science among the non-college bound in Pennsylvania. Unpublished Ph.D., The Pennsylvania State University, United States -- Pennsylvania.

Foundational Approaches in Science Teaching (FAST). (n.d.). Foundational Approaches in Science Teaching (FAST) submission to the Expert Panel on Mathematics and Science Education: Curriculum Research & Development Group, University of Hawai'i.

Fradd, S. H., Lee, O., Sutman, F. X., & Saxton, M. K. (2001). Promoting science literacy with English language learners through instructional materials development: A case study. Bilingual Research Journal, 25(4), 417-439.

Fraser, B. J., & Tobin, K. G. (Eds.). (1998a). International handbook of science education (Vol. 1). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Fraser, B. J., & Tobin, K. G. (Eds.). (1998b). International handbook of science education (Vol. 2). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Frederick, L. R., & Shaw, E. L., Jr. (1998, November 4-6). Examining the effects of science manipulatives on achievement, attitudes, and journal writing of elementary science students. Paper presented at the Annual Meeting of the Mid-South Educational Research Association, New Orleans, LA. (ERIC Document Reproduction Service No. ED 427 975).

Frederick, L. R., & Shaw Jr., E. L. (1999, November 17-19). Effects of science manipulatives on

achievement, attitudes, and journal writing of elementary science students revisited. Paper presented at the Annual Meeting of the Mid-South Educational Research Association, Point Clear, AL. (ERIC Document Reproduction Service No. ED436410).

Fredericksen, J. R., & White, B. Y. (2004). Designing assessments for instruction and accountability: An

application of validity theory to assessing scientific inquiry. In M. Wilson (Ed.), Towards coherence between classroom assessment and accountability. 103rd yearbook of the National Society for the Study of Education, Part II (pp. 74-104). Chicago: The University of Chicago Press.

Freedman, M. P. (1997). Relationship among laboratory instruction, attitude toward science, and achievement in science knowledge. Journal of Research in Science Teaching, 34(4), 343-357.

Gardner, M., Greeno, J. G., Reif, F., Schoenfeld, A. H., Disessa, A., & Stage, E. (Eds.). (1990). Toward a scientific practice of science education. Hillsdale, NJ: Lawrence Erlbaum Associates.

Initial Report


Garet, M. S., Birman, B. F., Porter, A., Desimone, L., Herman, R., & Yoon, K. S. (1999). Designing effective professional development: Lessons from the Eisenhower Program, Technical appendices.: Washington, D.C.: U.S. Department of Education. National Center for Education Statistics.

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-945.

Garner, L. C. (2004). Field trips and their effect on student achievement in and attitudes toward science: A comparison of a physical versus a virtual field trip to the Indian River Lagoon. Unpublished Ph.D., Florida Institute of Technology, United States -- Florida.

Gatlin, L. S. (1998). The effect of pedagogy informed by constructivism: A comparison of student achievement across constructivist and traditional classroom environments. Unpublished Ph.D., University of New Orleans, United States -- Louisiana.

Germann, P. J. (1994). Testing a model of science process skills acquisition: An interaction with parents' education, preferred language, gender, science attitude, cognitive development, academic ability, and biology knowledge. Journal of Research in Science Teaching, 31(7), 749-783.

Germann, P. J., Aram, R., & Burke, G. (1996). Identifying patterns and relationships among the responses of seventh-grade students to the science process skill of designing experiments. Journal of Research in Science Teaching, 33(1), 79-99.

Gibbons, S., Kimmel, H. S., & O'Shea, M. (1997). Changing teacher behavior through staff development: Implementing the teaching and content standards in science. School Science and Mathematics, 97(6), 302-309.

Glasson, G. E. (1989). The effects of hands-on and teacher demonstration laboratory methods on science achievement in relation to reasoning ability and prior knowledge. Journal of Research In Science Teaching, 26(2), 121-131.

Gobert, J., & Pallant, A. (2004). Fostering students' epistemologies of models via authentic model-based tasks. Journal of Science Education and Technology, 13(1), 7.

Greeno, J. G., Pearson, P. D., & Schoenfeld, A. H. (1996). Implications for NAEP of research on learning and cognition. Retrieved May 31, 2005

Greenwood, C. R., Terry, B., Marquis, J., & Walker, D. (1994). Confirming a performance-based instructional model. School Psychology Review, 23(4), 652-668.

Guthrie, J. T., Wigfield, A., & VonSecker, C. (2000). Effects of integrated instruction on motivation and strategy use in reading. Journal of Educational Psychology, 92(2), 331-341.

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal Of Physics, 66(1), 64-74.

Hamilton, L. S. (1998). Gender differences on high school science achievement tests: Do format and content matter? Educational Evaluation and Policy Analysis, 20(3), 179-195.

Hamilton, L. S., McCaffrey, D. F., Stecher, B. M., Klein, S. P., Robyn, A., & Bugliari, D. (2003). Studying large-scale reforms of instructional practice: An example from mathematics and science. Educational Evaluation and Policy Analysis, 25(1), 1-29.

Initial Report


Hamrick, L., & Harty, H. (1987). Influence of resequencing general science content on the science achievement, attitudes toward science, and interest in science of 6th grade students. Journal of Research In Science Teaching, 24(1), 15-25.

Hand, B., Wallace, C. W., & Yang, E. M. (2004). Using a science writing heuristic to enhance learning outcomes from laboratory activities in seventh-grade science: Quantitative and qualitative aspects. International Journal of Science Education, 26(2), 131-149.

Hapgood, S., Magnusson, S. J., & Palincsar, A. S. (2004). Teacher, text, and experience: A case of young children's scientific inquiry. The Journal of the Learning Sciences, 13(4), 455-505.

Hargrove, T. Y., & Nesbit, C. (2003). Science notebooks: Tools for increasing achievement across the curriculum. ERIC Digest. (ERIC Document Reproduction Service No. ED482720).

Harlen, W. (2004, May 11). Evaluating inquiry-based science developments. Paper presented at the

Evaluating of Inquiry-based Science meeting, Washington, D.C. (ERIC Document Reproduction Service No.).

Harwood, W. S., & McMahon, M. M. (1997). Effects of integrated video media on student achievement

and attitudes in high school chemistry. Journal of Research in Science Teaching, 34(6), 617-631.

Haukoos, G. D., & Penick, J. E. (1987). Interaction effect of personality-characteristics, classroom climate, and science achievement. Science Education, 71(5), 735-743.

Haussler, P., & Hoffmann, L. (2002). An intervention study to enhance girls' interest, self-concept, and achievement in physics. Journal of Research in Science Teaching, 39(9), 870-888.

Hayes, B. K., Goodhew, A., Heit, E., & Gillan, J. (2003). The role of diverse instruction in conceptual change. Journal of Experimental Child Psychology, 86(4), 253-276.

Henderson, D., Fisher, D., & Fraser, B. (2000). Interpersonal behavior, laboratory learning environments, and student outcomes in senior biology classes. Journal of Research in Science Teaching, 37(1), 26-43.

Hewson, P. W., Kahle, J. B., Scantlebury, K., & Davies, D. (2001). Equitable science education in urban middle schools: Do reform efforts make a difference? Journal of Research In Science Teaching, 38(10), 1130-1144.

Hiebert, J., & Stigler, J. W. (2000). A proposal for improving classroom teaching: Lessons from the TIMSS Video Study. Elementary School Journal, 101(1), 3-20.

Hill, M. D. (2002). The effects of integrated mathematics/science curriculum and instruction on mathematics achievement and student attitudes in grade six. Unpublished Ed.D., Texas A&M University - Corpus Christi, United States -- Texas.

Hilton, J. K. (2003). The effect of technology on student science achievement. Unpublished Ph.D., The Claremont Graduate University, United States -- California.

Hocutt, M. M. (2003). Comparing instructional methodologies in sixth-grade science: Traditional textbook, integrated science, and integrated science with technology enhancement. Unpublished Ph.D., The University of Alabama, United States -- Alabama.

Hoffer, T. B., & Gamoran, A. (1993). Effects of instructional differences among ability groups on student achievement in science and math. Dekalb, IL: Social Science Research Institute, Northern Illinois University. (ERIC Document Reproduction Service No. ED363509).

Initial Report


Hofstein, A., & Lunetta, V. N. (2004). The laboratory in science education: Foundations for the twenty-first century. Science Education, 88(1), 28-54.

Hopkins, K. S. (2001). The effects of computer simulation versus hands-on dissection and the placement of computer simulation within the learning cycle on student achievement and attitude. Unpublished Ed.D., Baylor University, United States -- Texas.

Houtz, L. K. E. (1995). Instructional strategy change and the attitude and achievement of 7th-grade and 8th-grade science students. Journal of Research In Science Teaching, 32(6), 629-648.

Howard, B. C., McGee, S., Shia, R., & Hong, N. S. (2001, April 10-14). Computer-based science inquiry: How componenets of metacognitive self-regulation affect problem-solving. Paper presented at the Annual Meeting of the American Educational Research Association, Seattle, WA. (ERIC Document Reproduction Service No.).

Hrabowski, F. A. (2003). Raising minority achievement in science and math. Educational Leadership,

60(4), 44-48.

Hsu, Y. S., & Thomas, R. A. (2002). The impacts of a web-aided instructional simulation on science learning. International Journal of Science Education, 24(9), 955-979.

Huffman, D., Goldberg, F., & Michlin, M. (2003). Using computers to create constructivist learning environments: Impact on pedagogy and achievement. Journal of Computers in Mathematics and Science Teaching, 22(2), 151-168.

Huffman, D., & Kalnin, J. (2003). Collaborative inquiry to make data-based decisions in schools. Teaching And Teacher Education, 19(6), 569-580.

Hunt, E., & Minstrell, J. (1998). A cognitive approach to the teaching of physics. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 51-74). Cambridge, MA: The MIT Press.

Hurley, M. M. (1999). Interdisciplinary mathematics and science: Characteristics, forms, and related effect sizes for student achievement and affective outcomes. Unpublished Ph.D., State University of New York at Albany, United States -- New York.

Ingram, S. J. (2003). The effects of contextual learning instruction on science achievement of male and female tenth-grade students. Unpublished Ph.D., University of South Alabama, United States -- Alabama.

Inverness Research Associates. (1999). The quality of the teaching of mathematics, science and technology in K-12 classrooms in New York state - A summary of findings. Retrieved May 31, 2005, from http://www.inverness-research.org/reports/ab1999-04_Rpt_NYSSI_MSTinNYState.htm

Irving, K. E., & Bell, R. L. (2004). Double visions: Educational technology in standards and assessments for science and mathematics. Journal of Science Education and Technology, 13(2), 255-266.

Jacobson, M. J., & Kozma, R. B. (Eds.). (2000). Innovations in science and mathematics education. Mahwah, NJ: Lawrence Erlbaum Associates.

Initial Report


Jafer, Y. J. (2003). The effects of computer-assisted instruction on fourth-grade students' achievement and attitudes toward desert issues. Unpublished Ph.D., Utah State University, United States -- Utah.

Johnson, E. G., & Owen, E. (1998). Linking the National Assessment of Educational Progress (NAEP) and the Third International Mathematics and Science Study (TIMSS): A technical report: Washington, D.C.: U.S. Department of Education. Office of Educational Research and Improvement.

Johnson, M. A., & Lawson, A. E. (1998). What are the relative effects of reasoning ability and prior knowledge on biology achievement in expository and inquiry classes? Journal Of Research In Science Teaching, 35(1), 89-103.

Johnson, R. T., Johnson, D. W., Scott, L. E., & Ramolae, B. A. (1985). Effects of single-sex and mixed-sex cooperative interaction on science achievement and attitudes and cross-handicap and cross-sex relationships. Journal of Research In Science Teaching, 22(3), 207-220.

Kahle, J. B., Meece, J., & Scantlebury, K. (2000). Urban African-American middle school science students: Does standards-based teaching make a difference? Journal of Research in Science Teaching, 37(9), 1019-1041.

Kaplan, D., & Elliott, P. R. (1997). A model-based approach to validating education indicators using multilevel structural equation modeling. Journal of Educational and Behavioral Statistics, 22(3), 323-347.

Kariuki, P., & Paulson, R. (2001, November 14-17). The effects of computer animated dissection versus preserved animal dissection on the student achievement in a high school biology class. Paper presented at the Annual Conference of the Mid-South Educational Research Association, Little Rock, AK. (ERIC Document Reproduction Service No. ED460018).

Keller, B. J. (1997). Effect of three different types of high school class schedules (traditional, rotating

block, and accelerated block) on high school biology achievement and on differences in science learning environments. Unpublished Ph.D., University of North Texas, United States -- Texas.

Kelly, M. P., & Staver, J. R. (2005). A case study of one school system's adoption and implementation of an elementary science program. Journal Of Research In Science Teaching, 42(1), 25-52.

Kennedy, M. M. (1999). Form and substance in mathematics and science professional development. NISE Brief, 3(2), 1-7.

Kim, J. J., Crasco, L. M., Smith, R. B., Johnson, G., Karantonis, A., & Leavitt, D. J. (2001). Academic excellence for all urban students: Their accomplishment in science and mathematics. Retrieved May 27, 2005, from http://www.systemic.com/pdfs/Booklet.pdf

Kim, N. B. (1997). A comparison of the effects of computer-enhanced with traditional instruction on the learning outcomes of high-school students in anatomy classes. Unpublished Ph.D., University of Pittsburgh, United States -- Pennsylvania.

Kimmel, H. S., Deek, F. P., Farrell, M. L., & O'Shea, M. (1999). Meeting the needs of diverse student populations: Comprehensive provessional development in science, math, and technology for teachers of students with disabilities. School Science and Mathematics, 99(5), 241-249.

Kimmel, H. S., Gibbons, S., & Oshea, M. (1997). Changing perspectives for professional development of K-12 science teachers. Abstracts Of Papers Of The American Chemical Society, 214, 282-CHED.

Initial Report


Kimmel, H. S., & O'Shea, M. (1999, November 10-14). Professional development and the implementation of standards. Paper presented at the 29th ASEE/IEEE Frontiers in Education Conference, San Juan, PR. (ERIC Document Reproduction Service No.).

Klahr, D., & Chen, Z. (2003). Overcoming the positive-capture strategy in young children: Learning about

indeterminacy. Child Development, 74(5), 1275-1296.

Klahr, D., & Chen, Z. (2004). Scientific reasoning: From formal models to classroom instruction. International Journal Of Psychology, 39(5-6), 4-4.

Klahr, D., Dunbar, K., Fay, A. L., Penner, D. E., & Schunn, C. D. (2000). Exploring science: The cognition and development of discovery processes. Cambridge, MA: The MIT Press.

Klahr, D., Fay, A. L., & Dunbar, K. (1993). Heuristics For Scientific Experimentation - A Developmental-Study. Cognitive Psychology, 25(1), 111-146.

Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction - Effects of direct instruction and discovery learning. Psychological Science, 15(10), 661-667.

Klein, S. P., Hamilton, L. S., McCaffrey, D. F., Stecher, B. M., Robyn, A., & Burroughs, D. (2000). Teaching practices and student achievement. Washington, D.C.: Rand Education.

Klentschy, M., Garrison, L., & Amaral, O. M. (2000). Valle Imperial project in science (VIPS): Four-year comparison of student achievement data 1995-1999. Retrieved May 27, 2005, from http://www.aea10.k12.ia.us/curr/science/science/ElCentroResearch.pdf

Kmetz, B. F. (2001). An analysis of factors correlated with the achievement of the goal standard for the science portion of the Connecticut Academic Performance Test. Unpublished Ed.D., University of Bridgeport, United States -- Connecticut.

Kopec, R. H. (2002). Virtual, on-line, frog dissection vs. conventional laboratory dissection: A comparison of student achievement and teacher perceptions among honors, general ability, and foundations-level high school biology classes. Unpublished Ed.D., Seton Hall University, College of Education and Human Services, United States -- New Jersey.

Krajcik, J., Mamlok, R., & Hug, B. (2001). Modern content and the enterprise of science: Science education in the twentieth century. In L. Corno (Ed.), Education across a century: The centennial volume. One hundredth yearbook of the National Society for the Study of Education, Part I (pp. 205-238). Chicago: The University of Chicago Press.

Kremer, P. L. (1983). Effects of individualized assignments on biology achievement. Journal of Research In Science Teaching, 20(2), 105-115.

Krueger, A., & Sutton, J. (Eds.). (2001). EDThoughts: What we know about science teaching and learning. Aurora, CO: Mid-continent Research for Education and Learning.

Lavoie, D. R. (1999). Effects of emphasizing hypothetico-predictive reasoning within the science learning cycle on high school student's process skills and conceptual understandings in biology. Journal of Research in Science Teaching, 36(10), 1127-1147.

Lawrence, L. C. (1997). The effects of an integrated Algebra 1/physical science curriculum on student achievement in Algebra 1, proportional reasoning and graphing abilities. Unpublished Ph.D., North Carolina State University, United States -- North Carolina.

Initial Report


Lawrenz, F., Huffman, D., & Robey, J. (2003). Relationships among student, teacher, and observer perceptions of science classrooms and student achievement. International Journal Of Science Education, 25(3), 409-420.

Lawrenz, F., Huffman, D., & Welch, W. (2001). The science achievement of various subgroups on alternative assessment formats. Science Education, 85(3), 279-290.

Lawrenz, F., & McCreath, H. (1988). Integrating quantitative and qualitative evaluation methods to compare two teacher inservice training programs. Journal of Research In Science Teaching, 25(5), 397-407.

Leach, J., Ametller, J., Hind, A., Lewis, J., & Scott, P. (2003, March). Evidence-informed approaches to teaching science at junior high school level: Outcomes in terms of student learning. Paper presented at the Annual Meeing of the National Association for Research in Science Teaching, Philadelphia, PA. (ERIC Document Reproduction Service No.).

Leach, J., & Scott, P. (2000, November). The concept of learning demand as a tool for designing teaching

sequences. Paper presented at the meeting Research-based Teaching Sequences, Universite Paris VII, France. (ERIC Document Reproduction Service No.).

Lee, O. (1999). Equity implications based on the conceptions of science achievement in major reform

documents. Review of Educational Research, 69(1), 83-115.

Lee, O., Eichinger, D. C., Anderson, C. W., Berkheimer, G. D., & Blakeslee, T. D. (1993). Changing middle school students conceptions of matter and molecules. Journal of Research in Science Teaching, 30(3), 249-270.

Lee, O., & Fradd, S. H. (1998). Science for all, including students from non-English-language backgrounds. Educational Researcher, 27(4), 12-21.

Lee, O., Hart, J. E., Cuevas, P., & Enders, C. (2004). Professional development in inquiry-based science for elementary teachers of diverse student groups. Journal Of Research In Science Teaching, 41(10), 1021-1043.

LeJeune, J. V. (2002). A meta-analysis of outcomes from the use of computer-simulated experiments in science education. Unpublished Ed.D., Texas A&M University - Commerce, United States -- Texas.

Lewis, E. L., & Linn, M. C. (1994). Heat energy and temperature concepts of adolescents, adults, and experts: Implications for curricular improvements. Journal of Research in Science Teaching, 31(6), 657-677.

Li, J. L., & Klahr, D. (2004). From psychology laboratory to urban classrooms: The training and transfer of the control of variable strategy. International Journal Of Psychology, 39(5-6), 93-93.

Li, W. G. (2005). General review of BES physics, achievement and future. International Journal Of Modern Physics A, 20(8-9), 1560-1567.

Lieberman, G. A., & Hoody, L. L. (1998). Closing the achievement gap: Using the environment as an integrating context for learning. Executive Summary. San Diego, CA: State Education and Environment Roundtable.

Lieberman, G. A., Hoody, L. L., & Lieberman, G. M. (2000). California student assessment project: The effects of environment-based education on Student achievement. San Diego, CA: State Education and Environment Roundtable (SEER).

Initial Report


Liew, C.-W., & Treagust, D. F. (1998, April 13-17). The effectiveness of predict-observe-explain tasks in diagnosing students' understanding of science and in identifying their levels of achievement. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA. (ERIC Document Reproduction Service No. ED420715).

Lightburn, M. E., & Fraser, B. J. (2002, April). Classroom environment and student outcomes associated

with using anthropometry activities in high school science. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA. (ERIC Document Reproduction Service No. ED465525).

Long, J. C., Okey, J. R., & Yeany, R. H. (1981). The effects of a diagnostic-prescriptive teaching strategy

on student achievement in biology. Journal of Research In Science Teaching, 18(6), 515-523.

Lott, G. W. (1983). The effect of inquiry teaching and advance organizers upon student outcomes in science-wducation. Journal of Research in Science Teaching, 20(5), 437-451.

Loucks-Horsley, S., Stiles, K., & Hewson, P. (1996). Principles of effective professional development for mathematics and science education: A synthesis of standards. NISE Brief, 1(1), 1-6.

Louden, C. K. (1997). Teaching strategies and student achievement in high school block scheduled biology classes. Unpublished Ph.D., The University of North Carolina at Chapel Hill, United States -- North Carolina.

Lyon, E. B. (1998). The effect of homogeneous and heterogeneous review pairs on student achievement and attitude when utilizing computer-assisted instruction in middle-level Earth science classes. Unpublished Ph.D., The University of Texas at Austin, United States -- Texas.

Lyons, R. P. (2001). Supporting adequate student outcomes: The effects of differential fiscal support on student achievement. Unpublished Ed.D., The University of Memphis, United States -- Tennessee.

Ma, X., & Ma, L. L. (2004). Modeling stability of growth between mathematics and science achievement during middle and high school. Evaluation Review, 28(2), 104-122.

Ma, X., & Wilkins, J. L. M. (2002). The development of science achievement in middle and high school - Individual differences and school effects. Evaluation Review, 26(4), 395-417.

Magnusson, S. J., Templin, M., & Boyle, R. A. (1997). Dynamic science assessment: A new approach for investigating conceptual change. The Journal of the Learning Sciences, 6(1), 91-142.

Maienschein, J. (2004). Laboratories in science education: Understanding the history and nature of science. Retrieved June 14, 2005, from http://www7.nationalacademies.org/bose/Jane_Maienschein_Presentation_Jun_04.pdf

Manhart, J. J., & Forsyth, R. A. (1999). Science achievement in the middle school years: IEA's Third International Mathematics and Science Study (TIMSS). Journal Of Educational Measurement, 36(1), 79-85.

Marchette, F. L. (2003). An investigation on impacts of scheduling configurations on Mississippi biology subject area testing. Unpublished Ed.D., The University of Southern Mississippi, United States -- Mississippi.

Marszalek, C. S. (1998). Effects on seventh-grade students' achievement and science anxiety of alternatives to conventional frog dissection. Unpublished Ed.D., Northern Illinois University, United States -- Illinois.

Initial Report


Martinez, A. (2002). Student achievement in science: A longitudinal look at individual and school differences. Unpublished Ed.D., Harvard University, United States -- Massachusetts.

Marx, R. W., Blumenfeld, P. C., Krajcik, J. S., Fishman, B., Soloway, E., Geier, R., et al. (2004). Inquiry-based science in the middle grades: Assessment of learning in urban systemic reform. Journal of Research in Science Teaching, 41(10), 1063-1080.

Matamoros, A. L., Lieberman, M. G., Morris, J. D., & Turner, S. F. (2001). Effects of curriculum course modifications on the science achievement of at-risk science students. The Journal of At-Risk Issues, 7(2), 41-47.

Matthews, D. R., & McLaughlin, T. F. (1994). Effects of learner-centered laboratory activities on achievement and students preferences in 2 high-school biology courses. Perceptual and Motor Skills, 78(1), 285-286.

Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? American Psychologist, 59(1), 12-19.

McCarthy, C. B. (2005). Effects of thematic-based, hands-on science teaching versus a textbook approach for students with disabilities. Journal of Research In Science Teaching, 42(3), 245-263.

McDermott, L. C., & Redish, E. F. (1999). RL-PER1: Resource Letter on Physics Education Research. American Journal of Physics, 67(9), 755-767.

McGilly, K. (Ed.). (1994). Classroom lessons: Integrating cognitive theory and classroom practice. Cambridge, MA: The MIT Press.

McManus, D. O., Dunn, R., & Denig, S. (2003). Effects of traditional lecture versus teacher-constructed & student-constructed self-teaching instructional resources on short-term science achievement & attitudes. American Biology Teacher, 65(2), 93-102.

McNeal, R. B. (1999). Parental involvement as social capital: Differential effectiveness on science achievement, truancy, and dropping out. Social Forces, 78(1), 117-144.

McRobbie, C. J., & Fraser, B. J. (1993). Associations between student outcomes and psychosocial science environment. Journal of Educational Research, 87(2), 78-85.

Miller-Whitehead, M. (2002, November 7). Making sense of class-size research, theory, and use. Paper presented at the Annual Meeting of the Mid-South Educational Research Association, Chattanooga, TN. (ERIC Document Reproduction Service No. ED470676).

Muller, P. A., Stage, F. K., & Kinzie, J. (2001). Science achievement growth trajectories: Understanding

factors related to gender and racial-ethnic differences in precollege science achievement. American Educational Research Journal, 38(4), 981-1012.

Myers, R. E., & Fouts, J. T. (1992). A cluster analysis of high school science classroom environments and attitude toward science. Journal of Research In Science Teaching, 29(9), 929-937.

National Research Council. (2000). How people learn: Brain, mind, experience, and school. Washington, D.C.: National Academy Press.

National Center for Education Statistics. (1998). Linking the National Assessment of Educational Progress (NAEP) and the Third International Mathematics and Science Study (TIMSS): A Technical Report (NCES 98-499): U.S. Department of Education. Office of Eductional Research and Improvement.

Initial Report


National Commission on Mathematics and Science Teaching. (2000). Before it's too late. Retrieved June 14, 2005, from http://www.ed.gov/inits/Math/glenn/report.pdf

National Research Council. (1996). National science education standards. Washington, D.C.: National Academy Press.

National Research Council. (2000). Inquiry and the National Science Education Standards: A guide for teaching and learning. Washington, D.C.: National Academy Press.

National Research Council. (2002). Learning and understanding: Improving advanced study of mathematics and science in U.S. high schools. Washington, D.C.: National Academy Press.

National Research Council, Hollweg, K. S., & Hill, D. (2003). What Is the Influence of the National Science Education Standards? Reviewing the evidence, a workshop summary. Washington, D.C.: The National Academies Press.

Nelson, D. I. (2003). What explains differences in international performance? TIMSS researchers continue to look for answers. CPRE Policy Briefs Retrieved June 9, 2005, from http://www.cpre.org/Publications/rb37.pdf

Nolen, S. B. (2003). Learning environment, motivation, and achievement in high school science. Journal of Research in Science Teaching, 40(4), 347-368.

Norman, O., Ault Jr., C. R., Bentz, B., & Meskimen, L. (2001). The Black-White "achievement gap" as a perennial challenge of urban science education: A sociocultural and historical overview with implications for research and practice. Journal of Research in Science Teaching, 38(10), 1101-1114.

O'Sullivan, C. Y., Lauko, M. A., Grigg, W. S., Qian, J., & Zhang, J. (2003). The nation's report card: Science 2000: U.S. Department of Education. National Center for Education Statistics.

Opfer, J. E., & Siegler, R. S. (2004). Revisiting preschoolers' living things concept: A microgenetic analysis of conceptual change in basic biology. Cognitive Psychology, 49(4), 301-332.

Orehowsky, W. (1999). The effect of laboratory-based instruction and assessment on student attitudes toward the laboratory experience and achievement in chemistry at the high school level. Unpublished Ed.D., Temple University, United States -- Pennsylvania.

Osman, M. E., & Hannafin, M. J. (1994). Effects of advance questioning and prior knowledge on science learning. Journal of Educational Research, 88(1), 5-13.

Palacio-Cayetano, J., Kanowith-Klein, S., & Stevens, R. (1999). UCLA's outreach program of science education in the Los Angeles schools. Academic Medicine, 74(4), 348-351.

Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition And Instruction, 1(2), 117-175.

Pallant, A., & Tinker, R. (2004). Reasoning with atomic-scale molecular dynamic models. Journal of Science Education and Technology, 13(1), 51.

Park, D.-Y. (2001). A study of Earth System Science in the Community (EarthComm) in terms of its congruency with the visions in the National Science Education Standards and its effectiveness in improving student learning. Unpublished Ph.D., The University of Iowa, United States -- Iowa.

Initial Report


Parke, H. M., & Coble, C. R. (1997). Teachers designing curriculum as professional development: A model for transformational science teaching. Journal of Research in Science Teaching, 34(8), 773-789.

Parker, V., & Gerber, B. L. (2000). Effects of a science intervention program on middle-grade student achievement and attitudes. School Science and Mathematics, 100(5), 236-242.

Parker, V. A., & Gerber, B. L. (2002). Performance-based assessment, science festival exhibit presentations, and elementary science achievement. Journal of Elementary Science Education, 14(1), 59-67.

Pearsall, M. K. (Ed.). (1992). Scope, sequence, and coordination of secondary school science: Volume II Relevant Research. Washington, D.C.: The National Science Teachers Association.

Penner, D. E., Giles, N. D., Lehrer, R., & Schauble, L. (1997). Building functional models: Designing an elbow. Journal of Research in Science Teaching, 34(2), 125-143.

Penner, D. E., & Klahr, D. (1996a). The interaction of domain-specific knowledge and domain-general discovery strategies: A study with sinking objects. Child Development, 67(6), 2709-2727.

Penner, D. E., & Klahr, D. (1996b). When to trust the data: Further investigations of system error in a scientific reasoning task. Memory & Cognition, 24(5), 655-668.

Penner, D. E., Lehrer, R., & Schauble, L. (1998). From physical models to biomechanics: A design-based modeling approach. The Journal of the Learning Sciences, 7(3&4), 429-449.

Pepper, W. L., Jr. (1999). The effect of using graphic organizers in the teaching of standard biology. Unpublished Ed.D., The University of Memphis, United States -- Tennessee.

Petrosino, A. J. (2004). Integrating curriculum, instruction, and assessment in project-based instruction: A case study of an experienced teacher. Journal of Science Education and Technology, 13(4), 447-460.

Porter, A., & Associates. (1994). Reform of high school mathematics and science and opportunity to learn. CPRE Policy Briefs Retrieved May 23, 2005, from http://www.cpre.org/Publications/rb13.pdf

Porter, A. C., Blank, R. K., Smithson, J. L., & Osthoff, E. (2005). Place-based randomized trials to test the effects on instructional practices of a mathematics/dcience professional development program for teachers. Annals Of The American Academy Of Political And Social Science, 599, 147-175.

Porter, A. C., & Smithson, J. L. (2001). Defining, developing, and using curriculum indicators. CPRE Research Report Series Retrieved May 19, 2005, from http://www.cpre.org/Publications/rr48.pdf

Powell, J. V., Aeby, V. G., & Carpenter-Aeby, T. (2003). A comparison of student outcomes with and without teacher facilitated computer-based instruction. Computers & Education, 40(2), 183-191.

Preece, P. F. W., & Brotherton, P. N. (1997). Teaching science process skills: Long-term effects on science achievement. International Journal of Science Education, 19(8), 895-901.

Project 2061 American Association for the Advancement of Science. (2000). Designs for science literacy. New York: Oxford University Press.

Pugh, K. J. (2002). Teaching for transformative experiences in science: An investigation of the effectiveness of two instructional elements. Teachers College Record, 104(6), 1101-1137.

Initial Report


Purser, R. K., & Renner, J. W. (1983). Results of two tenth-grade biology teaching procedures. Science Education, 67(1), 85-98.

Radford, D. L. (1988, April). Integrating process skills instruction into the traditional science curriculum. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, St. Louis, MO. (ERIC Document Reproduction Service No. ED291588).

Radford, D. L. (1998). Transferring theory into practice: A model for professional development for science

education reform. Journal Of Research In Science Teaching, 35(1), 73-88.

Rakow, S. J., & Walker, C. L. (1985). The status of Hispanic American students in science: Achievement and exposure. Science Education, 69(4), 557-565.

Reid-Griffin, A., & Carter, G. (2004). Technology as a tool: Applying an instructional model to teach middle school students to use technology as a mediator of learning. Journal of Science Education and Technology, 13(4), 495-504.

Reynolds, A. J., & Walberg, H. J. (1992). A structural model of science achievement and attitude - An extension to high school. Journal of Educational Psychology, 84(3), 371-382.

Rice, J. K. (2001). Explaining the negative impact of a the transition from middle to high school on student performance in mathematics and science. Educational Administration Quarterly, 37(3), 372-400.

Riley, J. P. (1986). The effects of teachers wait-time and knowledge comprehension questioning on science achievement. Journal of Research In Science Teaching, 23(4), 335-342.

Rivet, A. E., & Krajcik, J. S. (2004). Achieving standards in urban systemic reform: An example of a sixth grade project-based science curriculum. Journal of Research in Science Teaching, 41(7), 669-692.

Roberts, P. H. (1999). Effects of multisensory resources on the achievement and science attitudes of seventh-grade suburban students taught science concepts on and above grade level. Unpublished Ed.D., St. John's University (New York), United States -- New York.

Robertson, A. (2001). CASE is when we learn to think. Primary Science Review, 69, 20-22.

Rodriguez, M. C. (2004). The role of classroom assessment in student performance on TIMSS. Applied Measurement in Education, 17(1), 1-24.

Romance, N. R., & Vitale, M. R. (1992). A curriculum strategy that expands time for in-depth elementary science instruction by using science-based reading strategies: Effects of a year-long study in grade four. Journal of Research in Science Teaching, 29(6), 545-554.

Romance, N. R., & Vitale, M. R. (2001). Implementing an in-depth expanded science model in elementary schools: Multi-year findings, research issues, and policy implications. International Journal of Science Education, 23(4), 373-404.

Rose-Baele, J. S. (2003). Report of Fifth Grade Outcome Study, Science For All Students, 2001-2002. Retrieved June 6, 2005, from http://www.ccsdschools.com/administration/assessment/PIpage.html

Rosenbaum, H. (Ed.). (2000). Evaluation of event-based science project for the National Science Foundation ESI-9550498. Silver Spring, MD: Harvey Instruction.

Initial Report


Ross, J. A. (1990). Learning to control variables: Main effects and aptitude treatment interactions of two rule-governed approaches to instruction. Journal of Research in Science Teaching, 27(6), 523-539.

Ross, J. A., & Cousins, J. B. (1993). Enhancing secondary school students' acquisition of correlational reasoning skills. Research in Science & Technological Education, 11(2), n. p.

Ross, J. A., & Maynes, F. J. (1983). Experimental problem solving: An instructional improvement field experiment. Journal of Research In Science Teaching, 20(6), 543-556.

Roth, W. (1995). Authentic school science: Knowing and learning in open-inquiry science laboratories. Dordrecht, The Netherlands: Kluwer Academic Publishers.

Rubin, R. L., & Norman, J. T. (1992). Systematic modeling versus the learning cycle: Comparative effects on integrated science process skill achievement. Journal of Research In Science Teaching, 29(7), 715-727.

Ruby, A. M. (2001). Hands-on science and student achievement. Unpublished Ph.D., The RAND Graduate School, United States -- California.

Ruiz-Primo, M. A., Li, M., & Shavelson, R. J. (2001). Looking into students' science notebooks: What do teachers do with them? Retrieved July 6, 2005, from http://www.stanford.edu/dept/SUSE/SEAL/Reports_Papers/Cresst2001No2.pdf

Ruiz-Primo, M. A., Shavelson, R. J., Hamilton, L. S., & Klein, S. P. (2002). On the evaluation of systemic science education reform: Searching for instructional sensitivity. Journal of Research In Science Teaching, 39(5), 369-393.

Rutherford, F. J., & Ahlgren, A. (1990). Science for all Americans. New York: Oxford University Press.

Saunders, W. L., & Shepardson, D. (1987). A comparison of concrete and formal science instruction upon science achievement and reasoning ability of 6th grade students. Journal of Research in Science Teaching, 24(1), 39-51.

Saunders, W. L., & Young, G. D. (1985). An experimental-study of the effect of the presence or absence of living visual aids in high-school biology classrooms upon attitudes toward science and biology achievement. Journal of Research In Science Teaching, 22(7), 619-629.

Scantlebury, K., Boone, W., Kahle, J. B., & Fraser, B. J. (2001). Design, validation, and use of an evaluation instrument for monitoring systemic reform. Journal of Research In Science Teaching, 38(6), 646-662.

Schmidt, W. H., McKnight, C. C., Houang, R. T., Wang, H., Wiley, D. E., Cogan, L. S., et al. (2001). Why schools matter: A cross-national comparison of curriculum and learning. San Francisco: Jossey-Bass.

Schneider, L. S., & Renner, J. W. (1980). Concrete and formal teaching. Journal of Research in Science Teaching, 17(6), 503-517.

Schneider, R. M., Krajcik, J., & Blumenfeld, P. (2005). Enacting reform-based science materials: The range of teacher enactments in reform classrooms. Journal Of Research In Science Teaching, 42(3), 283-312.

Initial Report


Schneider, R. M., Krajcik, J., Marx, R. W., & Soloway, E. (2002). Performance of students in project-based science classrooms on a national measure of science achievement. Journal of Research in Science Teaching, 39(5), 410-422.

Schwartz-Bloom, R. D., & Halpin, M. J. (2003). Integrating pharmacology topics in high school biology and chemistry classes improves performance. Journal of Research in Science Teaching, 40(9), 922-938.

Shapka, J. D., & Keating, D. P. (2003). Effects of a girls-only curriculum during adolescence: Performance, persistence, and engagement in mathematics and science. American Educational Research Journal, 40(4), 929-960.

Shayer, M., & Adey, P. S. (1992). Accelerating the development of formal thinking in middle and high school students.3. Testing the permanency of effects. Journal of Research In Science Teaching, 29(10), 1101-1115.

Sherris, J. D. (1981, April 6). The effects of concept relatedness of instruction and locus of control orientation on the meaningful learning achievement of high school biology students. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, Grossinger, NY. (ERIC Document Reproduction Service No. ED205391).

Sherris, J. D., & Kahle, J. B. (1984). The effects of instructional organization and locus of control

orientation on meaningful learning in high school biology students. Journal of Research In Science Teaching, 21(1), 83-94.

Shields, P. M., Marsh, J. A., Adelman, N. E., & SRI International. (1998). Evaluation of NSF's Statewide Systemic Initiatives (SSI) Program: The SSIs' impacts on classroom practice. In National Science Foundation Directorate for Education and Human Resources (Ed.) (pp. 35): SRI International.

Shymansky, J. A., Hedges, L. V., & Woodworth, G. (1990). A reassessment of the effects of inquiry-based science curricula of the 60s on student performance. Journal of Research in Science Teaching, 27(2), 127-144.

Shymansky, J. A., Kyle, W. C., Jr., & Alport, J. M. (1983). The effects of new science curricula on student performance. Journal of Research in Science Teaching, 20(5), 387-404.

Shymansky, J. A., Yore, L. D., & Anderson, J. O. (2004). Impact of a school district's science reform effort on the achievement and attitudes of third- and fourth-grade students. Journal of Research in Science Teaching, 41(8), 771-790.

Singer, J., Marx, R. W., Krajcik, J., & Chambers, J. C. (2000). Designing curriculum to meet national standards. Retrieved May 31, 2005, from http://www-personal.umich.edu/~krajcik/Singer.Narst.pdf

Singh, K., Granville, M., & Dika, S. (2002). Mathematics and science achievement: Effects of motivation, interest, and academic engagement. Journal of Educational Research, 95(6), 323-332.

Smith, C., Wiser, M., Anderson, C. W., Krajcik, J., & Coppola, B. (2004). Implications of research on children's learning for assessment: Matter and atomic molecular theory. Retrieved June 14, 2005, from http://www7.nationalacademies.org/bota/Big%20Idea%20Team_%20AMT.pdf

Smith, J. R. (2004). Differences in student achievement between block period schools and nonblock period schools in the state of Mississippi. Unpublished Ph.D., The University of Southern Mississippi, United States -- Mississippi.

Initial Report


Snyder, R. C. (2001). The effect of science specialists versus generalist teachers on achievement and attitudes of students in the elementary grades. Unpublished Ph.D., University of Pittsburgh, United States -- Pennsylvania.

Songer, N. B., Lee, H. S., & McDonald, S. (2003). Research towards an expanded understanding of inquiry science beyond one idealized standard. Science Education, 87(4), 490-516.

Staver, J. R., & Walberg, H. J. (1986). An analysis of factors that affect public and private school science achievement. Journal of Research In Science Teaching, 23(2), 97-112.

Stemler, S. E. (2001). Examining school effectiveness at the fourth grade: A hierarchical analysis of the Third International Mathematics and Science Study (TIMSS). Unpublished Ph.D., Boston College, United States -- Massachusetts.

StohrHunt, P. M. (1996). An analysis of frequency of hands-on experience and science achievement. Journal of Research in Science Teaching, 33(1), 101-109.

Suits, J. P. (1997). Learning strategies and chemistry achievement. Abstracts Of Papers Of The American Chemical Society, 213, 739-CHED.

Supovitz, J. A., & Turner, H. M. (2000). The effects of professional development on science teaching practices and classroom culture. Journal Of Research In Science Teaching, 37(9), 963-980.

Sutman, F. X., Schumuckler, J. S., Hilosky, A. B., Priestley, H. D., & Priestley, W. J. (1996). Seeking more effective outcomes from science laboratory experiences (Grades 7-14): Six companion studies. Phildelphia, PA: Temple University. (ERIC Document Reproduction Service No. ED393703).

Tan, S. C. (2000). The effects of incorporating concept mapping into computer assisted instruction.

Journal Of Educational Computing Research, 23(2), 113-131.

Teitelbaum, P. (2003). The influence of high school graduation requirement policies in mathematics and science on student course-taking patterns and achievement. Educational Evaluation and Policy Analysis, 25(1), 31-57.

Tenenbaum, H. R., Snow, C. E., Roach, K. A., & Kurland, B. (2005). Talking and reading science: Longitudinal data on sex differences in mother-child conversations in low-income families. Applied Developmental Psychology, 26(1), 1-19.

Thomas, J. W., Bol, L., Warkentin, R. W., Wilson, M., Strage, A., & Rohwer, W. D. (1993). Interrelationships among students study activities, self-concept of academic ability, and achievement as a function of characteristics cf high-school biology courses. Applied Cognitive Psychology, 7(6), 499-532.

Thompson, J., & Soyibo, K. (2002). Effects of lecture, teacher demonstrations, discussion and practical work on 10th graders' attitudes to chemistry and understanding of electrolysis. Research in Science & Technological Education, 20(1), 25-37.

Timmerman, B., Strickland, D., Carstensen, S., & Singer, J. (2005). Using inquiry-based curricula to improve undergraduate science: Where are our efforts most fruitfully invested? Manuscript submitted for publication.

Tinoca, L., Lee, E., Fletcher, C., Barufaldi, J. P., & Meyer, J. (2004, April). From professional development for science teachers to student learning in science. Paper presented at the National Association for Research in Science Teaching, Vancouver, BC. (ERIC Document Reproduction Service No.).

Initial Report


Tobin, K. G., & Capie, W. (1982). Relationships between classroom process variables and middle-school science achievement. Journal Of Educational Psychology, 74(3), 441-454.

Toolin, R. E. (2004). Striking a balance between innovation and standards: A study of teachers implementing project-based approaches to teaching science. Journal of Science Education and Technology, 13(2), 179-187.

Toth, E. E., Klahr, D., & Chen, Z. (2000). Bridging research and practice: A cognitively based classroom intervention for teaching experimentation skills to elementary school children. Cognition And Instruction, 18(4), 423-459.

Triona, L. M., & Klahr, D. (2003). Point and click or grab and heft: Comparing the influence of physical and virtual instructional materials on elementary school students' ability to design experiments. Cognition And Instruction, 21(2), 149-173.

Troper, J. D. (1997). The effects of small-group discussion on students' learning and transfer of ideas in science. Unpublished Ph.D., University of California, Los Angeles, United States -- California.

Turpin, T. J. (2000). A study of the effects of an integrated, activity-based science curriculum on student achievement, science process skills, and science attitudes. Unpublished Ed.D., University of Louisiana at Monroe, United States -- Louisiana.

Valverde, G. A., & Schmidt, W. H. (2000). Greater expectations: Learning from other nations in the quest for 'world-class standards' in US school mathematics and science. Journal of Curriculum Studies, 32(5), 651-687.

Van Heuvelen, A. (1991). Overview, Case Study physics. American Journal of Physics, 59(10), 898-907.

Van Voorhis, F. L. (2003). Interactive homework in middle school: Effects on family involvement and science achievement. Journal of Educational Research, 96(6), 323-338.

VanTassel-Baska, J., Bass, G., Ries, R., Poland, D., & Avery, L. D. (1998). A national study of science curriculum effectiveness with high ability students. Gifted Child Quarterly, 42(4), 200-211.

Vasquez, J. A., Teferi, M., & Schicht. (2003). Science in the city: Consistently improved achievement in elementary school science results from careful planning and stakeholder inclusion. Science Educator, 12(1), 16-22.

Von Secker, C. E., & Lissitz, R. W. (1999). Estimating the impact of instructional practices on student achievement in science. Journal of Research in Science Teaching, 36(10), 1110-1126.

Wang, T., & Andre, T. (1991). Conceptual change text versus traditional text and application questions versus no questions in learning about electricity. Contemporary Educational Psychology, 16(2), 103-116.

Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. Retrieved May 31, 2005, from http://facstaff.wcer.wisc.edu/normw/WEBBMonograph6criteria.pdf

Weiss, I. R., Pasley, J. D., Smith, P. S., Banilower, E. R., & Heck, D. J. (2003). Looking inside the classroom: A study of K-12 Mathematics and Science Education in the United States. from http://www.horizon-research.com/insidetheclassroom/reports/looking/

Wen, M. L., Barrow, L. H., & Alspaugh, J. (2002, April 7-10). How does computer availability influence science achievement? Paper presented at the Annual Meeting of the National Association for

Initial Report


Research in Science Teaching, New Orleans, LA. (ERIC Document Reproduction Service No. ED468385).

Westby, C., Dezale, J., Fradd, S. H., & Lee, O. (1999). Learning to do science: Influences of culture and

language. Communication Disorders Quarterly, 21(1), 50-64.

White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3-118.

Willett, J. B., & Yamashita, J. J. M. (1983). A meta-analysis of instructional systems applied in science teaching. Journal of Research In Science Teaching, 20(5), 405-417.

Willson, V. L. (1983). A meta-analysis of the relationship between science achievement and science attitude: Kindergarten through college. Journal of Research In Science Teaching, 20(9), 839-850.

Windschitl, M., & Andre, T. (1998). Using computer simulations to enhance conceptual change: The roles of constructivist instruction and student epistemological beliefs. Journal of Research in Science Teaching, 35(2), 145-160.

Wise, K. C. (1996). Strategies for teaching science: What works? Clearing House, 69(6), 337-338.

Wise, K. C., & Okey, J. R. (1983). A meta-analysis of the effects of various science teaching strategies on achievement. Journal of Research in Science Teaching, 20(5), 419-435.

Yager, R. E. (1989). Comparison of standard student performance when science study is organized around typical concepts. Bulletin of Science, Technology, & Society, 9, 171-181.

Yager, R. E., & Weld, J. D. (1999). Scope, sequence and coordination: The Iowa Project, a national reform effort in the USA. International Journal Of Science Education, 21(2), 169-194.

Yang, E. M., Greenbowe, T., & Andre, T. (2003). Spatial ability and the impact of visualization/animation on learning electrochemistry. International Journal of Science Education, 25(3), 329-349.

Yang, E. M., Greenbowe, T., & Andre, T. (2004). The effective use of an interactive software program to reduce students' misconceptions about batteries. Journal Of Chemical Education, 81(4), 587-595.

Yasar, O. (2004). Computational math, science, and technology: A new pedagogical approach to math and science education. In Computational science and its applications - ICCSA 2004, Pt 3 (Vol. 3045, pp. 807-816).

Yasumoto, J. Y., Uekawa, K., & Bidwell, C. E. (2001). The collegial focus and high school students' achievement. Sociology of Education, 74(3), 181-209.

Yeany, R. H., Dost, R. J., & Matthews, R. W. (1980a). The effects of diagnostic-prescriptive instruction and locus of control on the achievement and attitudes of university students. Journal of Research In Science Teaching, 17(6), 537-545.

Yeany, R. H., Helseth Jr., E. A., & Barstow, W. E. (1980b, April). Interactive instructional video-tapes, scholastic aptitude, cognitive development and locus of control as variables influencing science achievement. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, Boston, MA. (ERIC Document Reproduction Service No. ED187532).

Initial Report


Yore, L. D. (1984). The effects of cognitive development and age on elementary students science achievement for structured inductive and semi-deductive inquiry strategies. Journal of Research In Science Teaching, 21(7), 745-753.

Yore, L. D. (1986). The effects of lesson structure and cognitive-style on the science achievement of elementary-school-children. Science Education, 70(4), 461-471.

Zady, M. F., Portes, P. R., & Ochs, V. D. (2003). Examining classroom interactions related to difference in students' science achievement. Science Education, 87(1), 40-63.

Zemelman, S., Daniels, H., & Hyde, A. (1998). Best practice in science. In Best practice: New standards for teaching and learning in America's schools (2nd ed., pp. 107-131). Portsmouth, NH: Heinemann.

Zender, G. A. (2002). An evaluation of a science professional development model: Examining participants' learning and use of new knowledge and skills, organizational support and change, and student learning outcomes. Unpublished Ph.D., The University of Akron, United States -- Ohio.

Zimmerman, C. (2000). The development of scientific reasoning skills. Developmental Review, 20(1), 99-149.

Zuniga, K., Olson, J. K., & Winter, M. (2005). Science education for rural Latino/a students: Course placement and success in science. Journal of Research in Science Teaching, 42(4), 376-402.

Initial Report


Appendix C

Literature Coding Document

Initial Report


CODING OF LITERATURE FOR SCIENCE INITIATIVE META-ANALYSIS 1-3 Study number & citation

4. Pub type: 1 Refereed Jour. Art. 2 Dissertation 3 Unpub. Report 4 Conference paper 5. Type of study: 1 Experimental (complete randomization) 2 Quasi –experimental (randomization used) 3 Quasi-experimental (no randomization) 4 Correlational Test Name #of

items Content Area(s)

6.Dep. Var.: 1 National Standardized Multiple Science content __________ ____ ______ 2 National Standardized Single Science content __________ ____ ______ 3 Local Standardized Multiple Science Content __________ ____ ______ 4 Local Standardized Single Science Content __________ ____ ______ 5Other type test __________ ____ ______

7. Ind. Var.: TRT. Name ___________________________________________________________

TRT. Description_______________________________________________________ TRT vs ______________________________________________________________ Course __________________

8. Length of TRT: Months Years

9. Setting & Characteristics: A. School (s): # of schools___, Selected at random? 1 Yes 2 No, Unit of analysis? 1 Yes 2 No

1 Public 2 Private, 1 Urban 2 Rural 1 Small (n=______), 2 Med (n=____), 3 Lrg (n=____) % Free Lunch

B. Students: # of students ___, Selected at random? 1 Yes 2 No Assigned at random? 1 Yes 2 No, Unit of analysis? 1 Yes 2 No 1 Mixed Gender 2 all Female 3 all Male 1 Same Grade 2 Multiple Grades Grade ____ n=___, Grade ___ n=__, Grade ___ n=___, Grade___ n=__ 1 Mixed Ethnicity 2 Single ethnic grp 1 Mixed Socio 2 Single Socio, Primary Socio Status_____________________

C. Teacher(s): # of teachers _________, 1 volunteer 2 selected

Teacher(s) description: Age/Exp Gender Certification

10. Study results: Effect size (s) ____, p(s) __, t(s) __ , F(s) ___, Eta Square(s) ___ Omega Square(s) ___ 11. Study rating: 1 True random assignment of schools/students to TRT and Control

2 Quasi-experimental with match of schools/students to achievement and demographics of comparison school

3 Quasi-experimental with covariate adjustment for prior achievement 4 Quasi-experimental comparison of schools/subjects based on a claim of similar 5 Quasi-experimental comparison of schools/subjects to region, state, or national

data 6 Quasi-experimental single-group pre-post comparison 7 Quasi-experimental treatment vs. control pre-posttest 8 Quasi-experimental multiple group ANOVA

12. Notes/comments:

Initial Report


Appendix D Treatment Description Category Classification Form

Initial Report


Study Treatment Category Rating

Study Number: __________ Judge: 1, 2, 3 1: unknown or not mentioned 2: mentioned 3: a small portion of the treatment 4: moderate portion of the treatment 5: largest portion of the treatment

Criteria 1 2 3 4 5 1. Questioning strategies

2. Focusing strategies

3. Manipulation strategies

4. Enhanced materials strategies

5. Testing strategies

6. Inquiry strategies

7. Enhanced context strategies

8. Instructional technology strategies

9. Direct instruction for teaching process skills

10. Collaborative learning strategies

Notes/comments:

Initial Report


Appendix E Treatment Description Summaries

Initial Report


Summary of Treatment Description

#1 Treatment Name web-based, constructivist-oriented homework assignment. Treatment description web-enhanced assignments covering DNA structure and function, Genetics and Microbiology Members of the experimental group participated in online homework activities for approximately eight weeks. The activities included participation in (a) online simulations, (b) asynchronous discussions, (c) Webquests, (d) cooperative concept map construction activities using synchronous chat technology, (e) online quizzes, and (f) personal, biweekly journaling activities. #4 Treatment Name Studio Model (both synchronously & asynchronously) Treatment description Studio Model and the use of supplemental on line materials/Tech & cooperative groups There are two characteristics surrounding the usage of the Studio Model. The first is the use of the technology (both synchronously and asynchronously) to give students the ability or means to see concepts or materials in multiple perspectives. The second is the integration of cooperative group activities. The cooperative group structure used for this study was the Student Team-Achievement Division (STAD). #6 Treatment Name The University of Alabama's Integrated Science (IS) Program enhanced with PowerPoint technology Treatment description The University of Alabama's Integrated Science Program enhanced with PowerPoint technology. Resources provided by the IS program include telecasts, teacher manuals, student books, science kits, and teacher professional development. #609 Treatment Name The University of Alabama's Integrated Science Program (IS) Treatment description The University of Alabama's Integrated Science Program (IS) Resources provided by the IS program include telecasts, teacher manuals, student books, science kits, and teacher professional development. #7 Treatment Name Computer-assisted instruction (CAI) Treatment description Computer software for animal and plants adaptation in desert. The Digital Field Trip to the Desert (2000) simulation software, designed by Digital Frog International Inc., on fourth-grade students’ achievement and attitudes toward desert issues. The software is designed as a tool that may be used by students individually or in small groups at their own pace with minimum amount of supervision. #21 Treatment Name Constructivist Based Instruction Treatment description Constructivist based instruction--application of students’ prior knowledge; human experiences and values, both culturally and socially determined, were incorporated into the classroom; student empowerment; critical thinking; opportunities were given students to exercise a degree of control over their learning environment; enable students to evaluate their own conceptual development.

Initial Report


#25 Treatment Name Web-Based Physics Software Program Treatment description Web-Based Physics Software Program "The Physics Classroom". This program consists three modules or sections: study topics, multimedia section, and quiz room. The multimedia section contains many animations. These animations visualize many ideas and misconceptions in order to make learning easy and dispel many preconceptions. The quiz room contains many quizzes related to study topics and gives deep understanding of each lesson. Each question in the quiz has a link to the related lesson if there is problem with understanding or solving this question. The web-based physics program was incorporated with the normal lecture. 30% of class time was allocated to using this tutorial program, and 70% of class time was used for normal lecture. #29 Treatment Name Integrated Science (IS) Curriculum Program Treatment description integrated, activity-based science curriculum. Resources provided by the IS program include telecasts, teacher manuals, student books, science kits, and teacher professional development. #30 Treatment Name Dynamic particle models--both static & dynamic Treatment description Computer assisted instruction Cooperative learning group analysis and teacher-directed class discussion. Dynamic class used both static and dynamic particle models and explanations in dynamic classes. Dynamic classes acted out particle motion. Dynamic classes watched particle movement and collisions illustrated with computer animations. #300 Treatment Name Dynamic particle models--both static & dynamic Treatment description Computer assisted instruction Cooperative learning group analysis and teacher-directed class discussion. Dynamic class used both static and dynamic particle models and explanations in dynamic classes. Dynamic classes acted out particle motion. Dynamic classes watched particle movement and collisions illustrated with computer animations. #32 Treatment Name Multiple teaching strategies Treatment description Multiple teaching strategies (teachers modify their teaching style to fit the many different types of students) The teachers were trained in the use of learning activities that require the utilization of four or more teaching strategies in class on a regular basis. Those strategies include activities such as simulations, role-playing, use of manipulative, computer, or other learning activities. #33 Treatment Name Multisensory instruction

Initial Report


Treatment description instructional strategy and student's perceptual preferences The multisensory unit included five instructional stations established in different sections of the classroom to allow students to learn by: (a) manipulating Flip Chutes, (b) using Electroboards, (c) assembling Task Cards, (d) playing a kinesthetic Floor Game, (e) reading an individual Programmed Learning Sequence. Audio tapes and scripts were provided at each section. Students circulated in groups of four from station to station. Students reviewed with the multisensory resources for 15 minutes prior to the posttest. #36 Treatment Name Laboratory experiences Treatment description the use of laboratory experiences to introduce classroom discussion and included lab-based performance activities to assess achievement The laboratory activities presented a question and required that the student provide an answer to that question based on the collection and analysis of data. Emphasis was placed on the need to record accurate descriptions and measurements and to analyze data mathematically and graphically. #38 Treatment Name Constructivist informed pedagogy Treatment description Constructivist informed pedagogy manipulative The constructivist teacher employed a lesson plan that was less didactic and used cooperative learning groups. Open-ended laboratory investigations, a high amount of technology, and a higher amount of student-initiated dialog among themselves as well as with the instructor. The constructivist teacher was also concerned with the identification of misconceptions, the use of probing questioning, and the learning cycle. Constructivist learning can be equated with the student’s gaining a higher level of inquiry regarding the work of science, connecting science with their own lives, and grasping main science concepts. #39 Treatment Name Computer-enhanced instruction (CEI) Treatment description Computer-enhanced instruction (CEI) using A.D.A.M. The Inside Story (1997) anatomy software. Students in the CEI class used A.D.A.M. The Inside Story (1997) software and worked in groups of three at the computers during normal class time. Students in both treatment and control classes received the same lectures, textbook reading assignments, study guides, prepared notes, and written assignments. In contrast to the control group class where the teacher functioned primarily as a disseminator of knowledge, the teacher functioned more as a facilitator in the CEI class. #41 Treatment Name Inquiry-based instruction Treatment description Predominately inquiry-based instruction Inquiry is a multifaceted activity that involves making observations; posing questions; examining books and other sources of information to see what is already known; planning investigations; reviewing what is already known in light of experimental evidence; using tools to gather, analyze, and interpret data; proposing answers, explanations, and predictions; and communicating the results. Teachers in non-block scheduled schools assigned more extended projects and student research; discuss their ideas and opinions. Teacher used cooperative learning and small group work more frequently in block scheduled classes.

Initial Report


#47 Treatment Name Interactive CD tutorial Treatment description CD tutorial for frog dissection Digital Frog is an interactive CD-ROM that incorporates full-motion video, animations, sounds, narration, in-depth text, full color photographs and a comprehensive workbook in three modules – Dissection; Anatomy, and Ecology. The dissection module uses a tutorial approach with the dissection proceeding in a step by step presentation manner. User can access multimedia files to view a frog dissection being performed, but does not actually “perform” a dissection themselves. #470 Treatment Name Desktop Microworld Treatment description CD tutorial for frog dissection The Desktop Microworld environment was built around Operation Frog on CD with other programs serving as auxiliary forms of instructional delivery. Operating Frog contains a tutorial simulation of the dissection of a frog, male or female. Operation Frog allows students to actually perform the dissection, not just view it. #48 Treatment Name Technology (GIS)-supported mapping Project Based Learning (PBL) unit Treatment description PBL, used GIS mapping technology for data analysis. The control group used a PBL science unit with paper mapping to support data analysis activities, while the experimental group used a PBL-GIS model. Experimental classrooms used ESRI’s ArcExplorer II as a desktop GIS application, with base data (roads, rails, hydrography, and airports) identical to the control group. All experimental classrooms had equal access to technology and related materials in a single, communal lab. #50 Treatment Name Scientific Writing Heuristic (SWH) Treatment description Students used the SWH student templates to guide their written work for laboratory activities. (writing format for 3 lab activities; meaning-making pedagogy) #500 Treatment Name Scientific Writing Heuristic and Textbook group (STG) Treatment description Students in these sections used the SWH for their laboratory activities; however, their written project to summarize the practical activities was different. They were asked to write their summary in the form of a textbook explanation for their peers. (writing format for 3 lab activities; meaning-making pedagogy) #57 Treatment Name Integrated Video Media Treatment description Integrated video media-World of Chemistry. These micro-units include teacher lesson guides associated with each 30-min World of Chemistry videotape designed to enable the teacher to stop the videotape approximately every 5-7 min for a teacher-student question-answer interaction time. #60 Treatment Name Design-Based Science (DBS)

Initial Report


DBS has much in common with other inquiry-based programs. Treatment description Students engaging in design projects to learn science: Extreme structures #600 Treatment Name Design-Based Science (DBS) DBS has much common with other inquiry-based programs. Treatment description Students engaging in design projects to learn science: Environmentally safe batteries #601 Treatment Name Design-Based Science (DBS) DBS has much common with other inquiry-based programs. Treatment description Students engaging in design projects to learn science: Safer celluar phones #64 Treatment Name HPD-LC (hypothetico-prediction-discussion learning cycle) Treatment description Adding a prediction/discussion phase to learning cycle. Two hypothetico-predictive problem sheets were prepared for each HPD-LC lesson. Each sheet required the student to make prediction in writing or graphically, and to support it with an explanatory reason (i.e., hypothesis), also in writing. #70 Treatment Name advance questioning Treatment description Basic lesson (B), Basic + orienting questions (B+Q), Basic+orienting questions+rationale (B+Q+R) conceptual orienting questions plus rationale. #75 Treatment Name Molecular Workbench (a two-dimensional molecular dynamics application written in Java software) molecular dynamic system vs. Pedagogica, a control environment Treatment description online molecular dynamics mode: States of Matter #750 Treatment Name Molecular Workbench molecular dynamic system Vs. Pedagogica, a control environment Treatment description online molecular dynamics mode: Atoms in Motion #78 Treatment Name Center for Learning Technologies in Urban Schools (LeTUS) Treatment description Inquiry-based and technology-infused curriculum units (standards based curriculum, prof. developed, 4 units taught): Air #780 Treatment Name Center for Learning Technologies in Urban Schools (LeTUS)

Initial Report


Treatment description Inquiry-based and technology-infused curriculum units (standards based curriculum, prof. developed, 4 units taught): Water #781 Treatment Name Center for Learning Technologies in Urban Schools (LeTUS) Treatment description Inquiry-based and technology-infused curriculum units (standards based curriculum, prof. developed, 4 units taught): Helmets #782 Treatment Name Center for Learning Technologies in Urban Schools (LeTUS) Treatment description Inquiry-based and technology-infused curriculum units (standards based curriculum, prof. developed, 4 units taught): Big Things #90 Treatment Name Concrete (hands-on) instruction vs. formal instruction Treatment description Concrete instruction was organized around the learning cycle approach and involved an emphasis upon hands-on activities; three phases: exploration, conceptual invention, and discovery (or application). #91 Treatment Name HASP (Hands-on Activity Science Program) inquiry-based program Treatment description Incorporated exemplary curriculum using modules, Full Option Science System (FOSS), incorporating manipulatives into elementary science #98 Treatment Name Science/Technology/Society (STS) format Treatment description Use societal issues as organizers for their study; study science in more community/citizenship activities rather than focus on knowledge and process #142 Treatment Name Kids as Global Scientists Weather (KGS) Program consists of a systematic, curricular approach to fostering students’ deep conceptual understanding of weather content Treatment description The use of a suite of learning tools designed specifically with inquiry science in mind. Tools include KGS curriculum, KGS software. (Urban teachers) #1420 Treatment Name Kids as Global Scientists Weather (KGS) Program consists of a systematic, curricular approach to fostering students’ deep conceptual understanding of weather content Treatment description The use a suite of learning tools designed specifically with inquiry science in mind. Tools include KGS curriculum, KGS software. (Maverick teachers)

Initial Report


#150 Treatment Name “Local Systemic Change Initiatives” (LSC) Treatment description Professional development (changing teaching practices, strengthening teacher content knowledge, and providing hands-on science materials for classrooms) , hands-on act, kits #166 Treatment Name Level of group cooperation Treatment description Cooperative goal structure (role assignment & nonrole assignment) For the role assignment group, each student was assigned a specific role (i.e., manager, investigator, and recorder) but students in both traditional and nonrole assignment groups were not assigned a role. #170 Treatment Name Detailed assignments Treatment description Detailed assignments (favoring field independence and induction) employed block diagrams and stepwise direction. Nondetailed assignments (favoring field dependence and deduction) virtually lacked these. #229 Treatment Name Alternative science teaching methods Treatment description Traditional (lecture, drill, practice) vs. teacher-constructed self-teaching manipulatives vs. student-constructed self-teaching manipulatives #245 Treatment Name Instructional Strategy change Treatment description Middle school interdisciplinary team strategy #247 Treatment Name Hands-on lab activities Treatment description Cell biology unit with hands-on lab activities #276 Treatment Name Hands-on laboratory method Treatment description Students manipulated lab apparatus in small groups #278 Treatment Name Resequencing general science content Treatment description Revising the order of textbook chapters, in order to clarify content structure and establish interrelationships among major concepts. (content arranged into interrelated pattern) #281 Treatment Name Wait-time after teacher questioning

Initial Report


Treatment description Teachers assigned wait-times of 1, 3, & 5 sec. #2810 Treatment Name Wait-time after teacher questioning Treatment description Teachers assigned wait-times of 1, 3, & 5 sec. #285 Treatment Name Single-sex & mixed sex cooperative interaction Treatment description In the cooperative conditions, subjects were instructed to work together as a group, completing the task as a group with all members sharing one set of materials and helping each other. They were instructed to make sure that each member was involved, and that they should check to make sure every member knew the materials and could explain the answers on the group’s data sheets. #369 Treatment Name Scope, Sequence and Coordination (SS&C) project Treatment description Extension from Chautauqua Model, constructivist learning model (CLM): invitation phase, exploration phase, coordination, implementation phase. integrated science; hands-on/minds-on ; problem centered #433 Treatment Name Concrete Instruction (learning cycle) First-hand experience Treatment description During the exploration phase of the learning cycle, students were usually provided with concrete materials and written directions for their use in gathering data about the concept to be learned. These direction sheets provided guidelines to help the students observe, compare, measure, and experiment, as they interacted with the selected concrete materials. #434 Treatment Name Concrete-the learning cycle inquiry Treatment description The instruction of the inquiry group was based upon the teaching concept designed and implemented by the Science Curriculum Improvement Study and called the learning cycle. Hands-on exploration, conceptual invention, discovery. #439 Treatment Name Science-Based Reading Strategies Treatment description Teachers implemented an in-depth science teaching program by expanding the lesson plans in their science textbook, Journeys in Science, to emphasize an integrated approach to science concept instruction through hands-on science activities, science process skills, and science textbook/ tradebook reading assignments. #448 Treatment Name Supported Inquiry Science (SIS), constructivist learning theory Treatment description

Initial Report


SIS principles: 1. provide a safe environment for expressing emerging or divergent ideas. 2. Design curricula and instruction to focus on a unifying concept. 3. Elicit students’ alternative conceptions and provide opportunities for them to examine conflicting evidence and reconstruct their ideas over time. 4. Use recursive, whole-class coaching to help students elaborate and revise their understanding. 5. Integrate opportunities for students to work with new material individually as well as in whole-class and small-group work. 6.~8. #451 Treatment Name Concept Relatedness Treatment description Instructional materials consisted of an explicit study, a reading supplement, and an activities supplement. The experimental study guide began with an introduction which explained how concepts and principles are important in the learning process and how to prepare a concept map. In addition, it included conceptual cues which reminded students of the importance of major concepts to related activities and readings. #452 Treatment Name Constructing Physics Understanding Project (CPU): Lead CPU Students Treatment description Computer-based modular curriculum activities: The CPU project produced modular content units and computer software, to support an environment where students, individually, in small groups, and as a whole class construct knowledge in physics. #4520 Treatment Name Constructing Physics Understanding Project (CPU): Beginning CPU studens Treatment description Computer-based modular curriculum activities: The CPU project produced modular content units and computer software, to support an environment where students, individually, in small groups, and as a whole class construct knowledge in physics. #455 Treatment Name Diagnostic-Prescriptive Teaching Treatment description Teacher-managed diagnostic-prescriptive assistance: Using progress checks (diagnostic tests) and review activities (prescriptive remediation) developed for this study. #457 Treatment Name Modified Mastery Learning strategy Treatment description Instruction for all classes was characterized by a blend of lecture, question-answer sessions, laboratory work, demonstrations, and audio-visual materials. Teacher-directed remediation and student-directed remediation (diagnostic quizzes and remediation activities). #467 Treatment Name Computer-animated dissection techniques Treatment description Use CD-ROM dissection tool "Dissection Works" #468 Treatment Name Full Option Science System (Foss)

Initial Report


Treatment description Use science manipulatives, hands-on/minds-on activities #475 Treatment Name Simulation before dissection (SBD) vs. dissection only (DO) Treatment description Combinations of actual & simulated frog dissection

Initial Report


Appendix F Comprehensive Meta-Analysis Output

Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Initial Report


Appendix G Regression Output

Initial Report


Regression without Length of Treatment

Variables Entered/Removedb

PublicationYear,DependentVariable,gradescode, PublishType,TreatmentCategories, StudyRating,Type ofstudy, TestContentcode

a

. Enter

Model1

VariablesEntered

VariablesRemoved Method

All requested variables entered.a.

Dependent Variable: Effect Sizeb.

Model Summary

.409a .167 .039 .722334Model1

R R SquareAdjustedR Square

Std. Error ofthe Estimate

Predictors: (Constant), Publication Year, DependentVariable, gradescode, Publish Type, TreatmentCategories, Study Rating, Type of study, Test Contentcode

a.

ANOVAb

5.451 8 .681 1.306 .261a

27.132 52 .52232.583 60

RegressionResidualTotal

Model1

Sum ofSquares df Mean Square F Sig.

Predictors: (Constant), Publication Year, Dependent Variable, gradescode, PublishType, Treatment Categories, Study Rating, Type of study, Test Content code

a.


Initial Report


Coefficientsa

.533 .874 .609 .545

.193 .183 .151 1.052 .297

.111 .235 .067 .474 .638

.012 .046 .036 .254 .801-.294 .185 -.233 -1.591 .118-.090 .070 -.181 -1.279 .207.028 .064 .064 .438 .663.040 .081 .066 .488 .628.062 .089 .117 .696 .489

(Constant)Type of studygradescodeTreatment CategoriesPublish TypeStudy RatingTest Content codeDependent VariablePublication Year

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: Effect Sizea.

Regression with Length of Treatment

Variables Entered/Removedb

Length ofTRT(month),TreatmentCategories,gradescode, Type ofstudy,PublishType,StudyRating,TestContentcode,DependentVariable,PublicationYear

a

. Enter

Model1

VariablesEntered

VariablesRemoved Method

All requested variables entered.a.


Initial Report


Model Summary

.441a .195 .018 .653986Model1

R R SquareAdjustedR Square

Std. Error ofthe Estimate

Predictors: (Constant), Length of TRT (month),Treatment Categories, gradescode, Type of study,Publish Type, Study Rating, Test Content code,Dependent Variable, Publication Year

a.

ANOVAb

4.243 9 .471 1.102 .383a

17.536 41 .42821.778 50

RegressionResidualTotal

Model1

Sum ofSquares df Mean Square F Sig.

Predictors: (Constant), Length of TRT (month), Treatment Categories, gradescode,Type of study, Publish Type, Study Rating, Test Content code, Dependent Variable,Publication Year

a.


Coefficientsa

.693 1.064 .652 .518

.177 .175 .157 1.012 .317-.092 .234 -.061 -.393 .696-.017 .045 -.057 -.387 .701-.132 .181 -.122 -.731 .469-.095 .082 -.191 -1.157 .254-.023 .072 -.056 -.314 .755.077 .108 .137 .714 .479.034 .093 .071 .364 .718.063 .041 .342 1.532 .133

(Constant)Type of studygradescodeTreatment CategoriesPublish TypeStudy RatingTest Content codeDependent VariablePublication YearLength of TRT (month)

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: Effect Sizea.

Initial Report


Appendix H Formulas

Initial Report


1. ScXXES CT −= where pre-tests were assumed to provide equivalent groups

XE = mean of experimental group (post-test) XC = mean of control group (post-test) SC = standard deviation of control group

2. Hedge’s d used in Comprehensive Meta-Analysis software

P

CT

SXXES −

= where ( ) ( )

211 22

2

−+−+−

=CT

CCTTP nn

SnSnS

3. n

tES 2= for nT=nC=n; n = sample size of each group

4. CT nn

tES 11+= for unequal ns

5. CT

CT

nnnn

FES+

=

6. ⎥⎦⎤

⎢⎣⎡

−−=

9431'N

ESES small sample size (N<30) adjusted with bias correction formula

7. ( )( )21

2

21

21

2 nnES

nnnn

se+

++

=

8. 2

1se

w =

9. Weighted average effect size: ( )∑

∑ ×=

wESw

ES

10. Stander Error (se) of the ES , ∑

=w

seES

1

11. Z-test for the ES , ES

seESZ =

12. 95% Confidence Interval

( )ES

seESLower 96.1−=

Initial Report


( )ES

seESUpper 96.1+=

13. Testing Homogeneity of Effect Size

WBT QQQ +=

( )∑=

−=k

iiiT ESESwQ

1

2.

( )2.... ESESwQ jjB −= ∑ 14. Failsafe N or Nfs

( )

C

Cfs ES

ESESNN −= where N= # of studies in the meta-analysis

=ES average ES for the studies in the meta-analysis

=CES criterion ES

Date post:	10-Feb-2017
Category:	Documents
Upload:	lehanh
View:	220 times
Download:	0 times

Meta-Analysis of National Research Regarding Science Teaching

Documents