Publications and Other Resources Resulting from a Synthesis of Research on the Impact of Inquiry Science Instruction
Project Update as of April 2007
The inquiry synthesis study has been completed and a final report has been prepared and sent to our funder, the National Science Foundation. We are in the process of writing articles for submission to peer-reviewed journals. Once we are informed regarding the results of these reviews, we will be better prepared to share the findings with our practitioner, policy, and research colleagues in a variety of venues. Until then, we have prepared an executive summary of main findings that can be obtained by contacting Daphne Minner at dminner@edc.org. For more information about the conceptual and methodological aspects of the project, please review the six technical reports. General overview information about the project can be found below.
Introduction
The Center for Science Education (CSE) at Education Development Center, Inc., (EDC) has just completed a four-year study funded by the National Science Foundation that addressed the question: what is the impact of inquiry science instruction on student outcomes compared with the impact of other instructional strategies and approaches? This project synthesized research that met the following criteria:
- Was completed between 1984 and 2002
- Was conducted with K–12 students
- Had at least one research question that was about the effect of an instructional intervention in science on student outcomes
- Focused on at least one student instructional intervention in science;
- Described the student instructional intervention with sufficient specificity; and
- Reported student outcomes
The synthesis included 138 studies with a dependent variable of science subject matter content knowledge. Of these, 105 were conducted in the United States; 42 studies focused on elementary grades; 49 on middle grades; and 47 on high school grades. There were 339 schools, 589 classrooms, and 13,066 students included as subjects of these studies; however, a number of studies did not report on these characteristics, and so these totals underreport the actual numbers. Of the 138 studies, 83 (60%) focused on physical science; 35 (25%) focused on life science; and 16 (12%) focused on earth/space science. A very small number (3%) examined the multiple content areas of physical and life science; and physical and life and earth science. The dependent variable—science-subject-matter content knowledge—was divided into six different finding types, which were expressed in physical science, life science, or earth/space science. The finding types are listed below, followed by the number of studies that had such a finding type as its dependent variable (some studies had more than one finding type):
- Understanding(s) related to science facts and vocabulary (104 studies, or 60%)
- Understanding(s) related to science concepts (28 studies, or 16%)
- Understanding(s) related to science principles and theories (19 studies, or 11%)
- Retention of science facts and vocabulary (9 studies, or 5%)
- Retention of science concepts (11 studies, or 6%)
- Retention of science principles and theories (3 studies, or 2%)
Purpose of Study
In the past decade, organizations such as the National Research Council (NRC) and the American Association for the Advancement of Science (AAAS) have made significant commitments to improving science education. A common theme among these efforts is the presence of inquiry instructional strategies. In addition to these large-scale national efforts, many school districts across the country have devoted their own resources to reforming their science programs by adopting hands-on science materials and promoting the pedagogy of inquiry.
Given the recent increased attention to student performance on high-stakes tests due to the No Child Left Behind legislation, the need for evidence about effective educational practices is pressing. Thus, in light of the investments made to date in the pedagogy of inquiry, it is essential that a critical question be asked: “What do we know about the impact of inquiry science programs on student outcomes in science?” Almost 20 years after the latest meta-analysis of the effect of inquiry science teaching (Bredderman, 1983; Shymansky, Kyle, & Alport, 1983), it is essential to reassess the accumulated evidence of impact in order to draw conclusions that are relevant for today’s educators and policy makers.
This study differs from the previous synthesis in several ways:
- It includes studies that both employ a range of research methods and examine a variety of student outcomes.
- It includes studies that investigate the impact of a range of inquiry science teaching strategies, not necessarily those associated with particular curricula.
- It will examine the extent and nature of the inquiry science instruction and whether its impact varies across student characteristics such as gender, race, and/or socio-economic status.
The study will provide a full account of what the field has learned since 1984 about the impact of inquiry science on student outcomes. These findings will be systematically disseminated to researchers, policy makers, administrators, teachers, parents, and the public at large to help inform their decisions about science education.
Bredderman, T. (1983). Effects of activity-based elementary science on student outcomes: A quantitative Synthesis. Review of Educational Research, 53 (4), 499–518.
Shymansky, J., Kyle, W., and Alport, J.
(1983). The effects of new science curricula on student performance. Journal of Research in Science Teaching, 20(5),
387–404.
Phases of the Study
This project included three broad phases.
- Phase I—Report
Collection: This phase involved conducting a search for
research reports that met the criteria.
The search
was shaped by the objective of “casting a broad net” in
order to identify and retrieve all reports that could potentially
be part of the synthesis (a description of this process
can be found in Technical Report 1: Generating the
Synthesis Sample of Studies).
There is a purposeful distinction made throughout this technical report between a “report” and a “study.” Often authors disseminate multiple reports from a single study. From Phase I through Phase II–stage one, reports are the primary unit of interest. From Phase II, stage two through Phase III, studies (which could be represented by multiple reports) are the primary unit of interest.
- Phase II—Coding Process: Once retrieved, reports entered this phase, which comprised
three stages.
- In stage one—inclusion/exclusion—each report was carefully screened and coded to determine if the inclusion criteria were met. These criteria include the following: the report focused on a science instructional intervention; the report included student outcomes; the outcomes were directly related to the instructional intervention; the report was completed between January 1, 1984, and December 31, 2002; the study was conducted with K–12 students; and the intervention was described with sufficient specificity. Reports that met all of these criteria were included in the synthesis dataset. Details about the development of the codebook used for the inclusion/exclusion coding and the process for establishing interrater agreement can be found in Technical Report: Operationalizing the Inclusion/Exclusion Coding Process (in development). The descriptive information about the initial synthesis sample that resulted from the inclusion/exclusion coding process can be found in Technical Report: Describing the Landscape (in development).
- Stage two of the coding process, inquiry instruction description, involved developing and applying a detailed coding schema to describe the instructional intervention. This descriptive methodology is the subject of Technical Report: Operationalizing the Inquiry Science Instruction Coding Process (in development). Technical Report 2: Conceptualizing Inquiry Science Instruction, articulates the theoretical underpinnings of the coding protocol for describing inquiry instruction.
- Stage three of coding, research rigor, context, and study findings, involved capturing all other relevant information about the methodological integrity of the research, the context of the study, covariates, comparison treatments, and study findings. The third stage of coding will be the subject of future technical reports produced over the next year.
Each stage of Phase II afforded an opportunity to exclude a report or study due to misalignment with the data requirements of the synthesis. To remain included, reports/studies were required to include the appropriate variables, to provide sufficient information to code the variables of interest, to present evidence of a minimum threshold for inquiry science instruction, and to exhibit a minimum threshold for research rigor. While the search for reports in Phase I was broad to better ensure that all eligible studies would be found, it was understood that a broad search would result in the collection of some number of reports that would not meet the minimum criteria. Thus, this multi-step process was designed to refine and narrow the sample of studies included in this synthesis in order to ensure that the analyses would be based on only those studies that adequately reported clear descriptions of an inquiry science intervention and were conducted with sufficient rigor to allow for drawing credible conclusions.
- Phase III of the project includes the analysis and dissemination of results.
Contact Us
Contact us at cse_evaluation_research@edc.org for more information.
This material is based upon work supported by the National Science Foundation under Grant No. 0101766.
