Researchers from the National Center on School Choice engaged in a lively discussion and came away with helpful suggestions during a roundtable session at the recent American Educational Research Association annual meeting in Denver. The discussion centered on new findings from an ongoing NCSC project that looks at how factors inside schools – from leadership and organization to teaching conditions and practices – influence student achievement gains in mathematics. The project, “Opening the Black Box in Choice and Regular Public Schools: A Study of What Makes Schools Work,” compares findings for schools of choice and traditional public schools.
The project involves extensive surveying of teachers on a range of topics, including the instructional climate in their schools, which topics they taught, the cognitive demands of those topics, and characteristics of the school’s leadership. Principals also have been surveyed on topics related to organization and academics. Charter and traditional public schools were matched on demographic characteristics, school size and distance from each other.
Project researchers reported findings from two papers at AERA. The first paper examines differences between charter and public school teachers in instructional practices. The study investigates several aspects of mathematics instruction for the teachers surveyed: breadth and depth, cognitive complexity, and alignment to standards. Using an instrument called the Surveys of Enacted Curriculum, the study asked teachers about the mathematics topics they taught and the cognitive demand expected for each topic. The survey seeks an intersection between the two aspects of instruction. For example, teachers were asked about their level of emphasis on a range of topics, including multiple-step equations, inequalities, linear equations, lines/slope and intercept, operations on polynomials, and quadratic equations. They were then asked to rate the level of work expected on a scale of increasing complexity: memorize, perform procedures, communicate understanding, solve problems, or conjecture/generalize/prove. The study found few differences, on average, between teachers in charter and traditional public schools in the content, cognitive complexity, and the alignment of their instruction to standards from the National Council of Teachers of Mathematics.
The other paper analyzes the array of professional development that teachers used at their schools and investigates the relationship between these programs and the teachers’ mathematics instruction. Teachers were asked about “core” features of their professional development, including duration of the programs; the content-focus of the professional development; the degree to which the sessions were presented in non-traditional, or reform, formats; whether the sessions included most or all teachers in a teacher’s school or grade level; the degree to which the professional development was consistent with other activities in the school; and whether the sessions included activities that allowed participants to be actively involved in their own learning, such observing demonstrations of teaching techniques. The paper writes that several features of professional development were associated with a reduced use of activities using lower levels of cognitive demand and increased use of activities requiring high levels of cognitive demand.
The session discussant, Andrew Porter of the University of Pennsylvania, reminded the audience that theory holds that charter schools can improve educational quality in two ways: by providing schools that are innovative and/or better; and by creating a competitive environment that will prod traditional public schools to improve. If the latter has occurred in the locations covered by this project, maybe it’s not surprising that more differences did not emerge between charter and traditional public schools, said Porter, who was a co-developer of the Surveys of Enacted Curriculum. He noted that the team will learn more about similarities and differences as they continue to “look under the hood” and analyze the educators’ survey responses to understand what is happening inside the schools. Porter also asked whether the similarities found between the two sectors, on average, might also be due to matching techniques used by the project. Even though the schools are matched mainly on demographics, do these matches still reduce the likelihood of differences in instructional practices?
Read more about Opening the Black Box in Choice and Regular Public Schools: A Study of What Makes Schools Work.
