Find out about new CSRQ Center resources!
Send us your email address to sign up for our email updates.
If you have any questions or would like more information about the CSRQ Center, please contact us:
The CSRQ Center Reports will provide consumer friendly reviews and guidance on the effectiveness and quality of the leading comprehensive school reform (CSR) programs. The reports will be designed to give education consumers the in-depth information they need to make the best possible adoption decisions to meet locally defined needs.
Each report will provide basic information on the CSR program, including the program's mission and focus, year introduced in schools, grade levels served, number of schools served, and costs. The reports will evaluate the following dimensions of quality:
Whenever possible, the CSRQ Center Reports will offer information on program results for specific student groups or specific types of school settings.
In addition, the CSRQ Center will work with The Finance Project to review two additional dimensions of quality, beginning in September 2005:
The Finance Project is a non-profit policy research, technical assistance, and information organization whose mission is to support decision making that produces and sustains good results for children, families, and communities. Since its establishment in 1994, The Finance Project has become an unparalleled resource on issues related to the financing of education, family and children's services, and community development. Their work informs the development of effective policies, programs, and system reform efforts.
July 2004 Quality Review Tools released
January 2005 CSRQ Reports on Elementary School CSR Programs released
September 2005 CSRQ Reports on Education Management Organizations released
September 2006 CSRQ Reports on Middle and High School CSR Programs released
September 2006 Updated CSRQ Reports on Elementary School CSR Programs released
Additionally, in October 2004, the CSRQ Center will release a report called Works in Progress: A Report on Middle and High School Improvement Programs. This report will not use the Quality Review Tools to present a full-scale review of CSR. Instead, this report will discuss challenges facing middle and high school educators, highlight current middle and high school reform efforts, characterize existing evidence on these reforms, and suggest next steps for research and practice.
If, as a country, we are to realize the potential offered by comprehensive school reform, decision makers will need help in sorting through competing claims made by researchers and program providers about what works. The CSRQ Center Reports will provide scientifically based, consumer-friendly reports on program quality and effectiveness and the guidance needed to use this evidence to make effective, locally defined choices.
The promise of CSR programs and the school wide reform that they support is that they are research-based and provide the training and other supports needed to encourage a coordinated approach to improvement that addresses curriculum and instruction, professional development, leadership, parental and community involvement, and other components needed for student success. Since federal CSR funds became available in 1998, schools nationwide have adopted more than 700 different CSR reforms. Research evidence to date indicates that some of these programs are more effective than others and that results vary greatly—even with the most effective programs—depending on the quality of implementation.
Whether a school adopts a program that offers a comprehensive package of practices or decides to build its own from individual research-proven components, decision makers need reliable information to help them answer essential questions: Which of these programs work well to raise student achievement or accomplish other important student outcomes? Do some of them work at all? Without the CSRQ Center Reports, those most directly responsible for improving education—state officials, school board members, administrators, and teachers—and those concerned about its success—educators, parents, policymakers, and the public—have few resources at their disposal to answer these questions
The CSRQ Center Reports will be guided by the CSRQ Center's Quality Review Tool, or QRT, which provides the criteria for independent, fair, and credible program reviews. To ensure that the QRT is valid, reliable, and credible, the QRT development process involved several steps. First, CSRQ Center staff developed review frameworks in consultation with some of the nation's most respected education researchers, program evaluators, and school improvement experts. Then, the QRT was reviewed, revised, and, ultimately, endorsed, by the CSRQ Center's Advisory Committee, a nationally respected panel of experts, that includes leading education practitioners, methodologists, and researchers from a variety of fields, including education, sociology, medicine, psychology, and economics. The QRT will be updated regularly as new evidence and improved methodologies become available.
The Quality Review Tool will be available on this website in July 2004.
The Quality Review Tool, or QRT, is a set of forms, rubrics, and evaluation criteria that have been carefully designed to guide the CSRQ Center reviews of CSR reform programs. The tool is intended to make the review process clear and transparent. The QRT was designed by CSRQ Center staff and reviewed by an Advisory Committee consisting of national experts in the field of schoolwide reform, program evaluation, methodology, and education. The QRT also draw on other efforts to conduct rigorous research reviews, including the What Works Clearinghouse.
The QRT review process is divided into three parts, and each part guides a distinct phase of the review process.
For a graphic overview of the QRT review process, click here. (This is a 36.7 KB Microsoft PowerPoint document. Get the free PowerPoint viewer for Windows or Mac.)
Part 1 of the QRT is an information cataloguing system that strives to acquire as much information as possible about all programs available. It consists of a multifaceted process for collecting and verifying information about all programs being considered for inclusion in the review of programs. The process acquires information about programs from literature reviews, interviews with program staff, and interviews with staff at schools implementing the programs.
Once the CSRQ Center has collected enough formative information about the CSR reform program under review, CSRQ reviewers analyze the program's evidence of effectiveness and research base. QRT Part 2 examines the rigor of the research design of each individual study on a CSR program's effectiveness. QRT Part 2 does not examine the strength of a CSR program's impact; but rather, it judges the quality of the research design.
CSRQ reviewers use Part 2 to decide whether a study is strong or conclusive enough to be considered in Part 3 of the QRT review. To do so, CSRQ reviewers examine the study to determine whether it meets QRT Part 2 standards for internal and external validity; independence of the researchers; face and psychometric validity of the outcome measures; size of program effects; and other quality indicators. Only research studies that proceed to QRT Part 3 count toward a program's ratings in the CSRQ Center reports.
QRT Part 3 is broken down into rubrics that establish standards against which research on a program's impact will be examined and rated. If CSRQ reviewers deem the rigor of a study's research design to be strong or conclusive, using QRT Part 2, the study then proceeds to QRT Part 3. Using QRT Part 3, reviewers look across studies on a CSR reform program and rate the cumulative evidence on a program as strong, moderate, limited, weak, or nonexistent. Using research that meets the standards set forth in QRT Part 2, these rubrics will help evaluate the extent to which a program can demonstrate positive impact in the following categories:
In addition, the CSRQ Center will work with The Finance Project to develop rubrics for two additional dimensions of quality and apply these standards to reviews beginning in mid-2005:
In total, the CSRQ Center will review approximately 50 to 70 CSR programs that have been widely implemented, meaning that the program is in use in at least three states and five schools total. The programs to be reviewed include
CSR programs will be identified through an extensive search of existing data on CSR implementation, including the Southwest Educational Development Laboratory's CSR Awards Database, the National Clearinghouse on Comprehensive School Reform, and the Northwest Regional Education Lab's Catalog of School Reform Models.
The CSRQ Center will also accept nominations of programs to review by email. To nominate a program, please send us the program's name, the program's website, and the program developer's contact information (mailing address, phone number, and email address).
Some education decision makers may be interested in CSR programs, including new or smaller programs, that have not yet been reviewed by the CSRQ Center Reports. Thus, the CSRQ Center will make a nonevaluative CSR Program Registry available so that program developers have the opportunity to share their own information about programs not included in the CSRQ Center Reports. For more information about the Program Registry, click here.
Our Reports
