NICHCY: Creating Environments That Work for All Youth: Increasing the Use of Evidence-Based Strategies by Special Education Teachers

By Krista Kutash and Albert J. Duchnowski

Using instructional strategies grounded in strong empirical foundations will improve the educational outcomes of students in both general and special education. The President’s Commission on Excellence in Special Education (PCESE) recommends the replacement of an established "culture of compliance" with a "culture of results" built on improved instruction supported by research and ensured by increased accountability (PCESE, 2002). One issue of The Journal of Special Education highlighted a series of research-based instructional practices for children with disabilities (Cook & Schirmer, 2003). The special education literature contains several research syntheses and meta-analyses of evidence-based practices (e.g., Forness, Kavale, Blum, & Lloyd, 1997; Gersten, Schiller, & Vaughn, 2000). However, the consensus in the field is that there is a vast gap between practices proven to be successful through research and what is practiced in our schools (Greenwood, 2001). The failure to implement and sustain effective practices in the classroom has been offered as a major explanation for the poor outcomes for special education students (Greenwood & Abbott, 2001; Landrum, Tankersley, & Kauffman, 2003).

With a goal of increasing the use of evidence-based practices in special education programs and improve student outcomes, a research demonstration project was developed through a unique partnership of special educators, parents, administrators, and investigators. This brief reports on the method, implementation, and initial findings from this project.

At the beginning of the project, partnership members held a series of planning meetings. The participants discussed overall goals and adopted a plan of operation. The first task consisted of developing teacher-friendly manuals that would help implement evidence-based practices in class. Completing this task required a discussion of what constitutes evidence-based practice. Participants presented their perspectives and identified potential strategies; the researchers made sure that consistent scientific evidence guided the project. As the discussions proceeded, researchers gathered information to verify that each strategy was evidence-based and ultimately improved a desired outcome.

Subsequently, the group agreed to the following topics for effective Effective Strategies Manuals (ESMs): (a) enhancing reading comprehension, (b) formative evaluation, (c) positive behavior supports, and (d) family involvement. A significant body of empirical literature supports the effectiveness of each of these topic areas (Duchnowski, Kutash, Sheffield, & Vaughn, in press).

To enhance reading comprehension, the group agreed to adapt a reading program that the participating school district was using and for which the teachers had been trained. Creating Reading Independence Through Student Strategies (CRISS) is a combination of direct and strategies instruction (Santa, Havens, & Maycumber, 1996). Programs that use a combination of these techniques have been found to have an average effect size of .84 when used to instruct children who have reading problems (Swanson, 2000). In developing the first ESM, core components from the CRISS manual were used to produce a more teacher-friendly guide to assist teachers in implementing evidence-based practices in the special education program. The reading comprehension ESM included the use of effective strategies such as mnemonics, selective highlighting and underlining, finding main ideas, and using graphic organizers (Forness et al., 1997).

Based on the literature review, a formative evaluation was selected as a strategy for inclusion in the project. However, the term "formative evaluation" was considered by the teachers to be technical jargon and was replaced by "providing academic feedback." The ESM for this strategy contained sample charts and graphs that the teachers could customize and students could use to monitor their progress on a frequent basis (Forness et al., 1997).

Strategies using positive behavior supports were aimed at reducing challenging behaviors and increasing desirable social and adaptive skills in students. These strategies included (a) emphasizing strengths, (b) providing accommodations in the classroom environment, and (c) developing effective classroom rules (Carr et al., 2002). This ESM emphasized the findings in the literature that identify students’ inappropriate behaviors as often being goal directed to escape or avoid instructional activity that the student may not understand. This necessitates the teacher examining more carefully the student’s skill level and assessing the need for accommodations or alternative instructional strategies. In addition, by linking the ESMs in reading and formative evaluation to the behavior of students, the project promoted a comprehensive approach rather than addressing behavior issues in a manner unconnected to academics.

All the partners agreed that family involvement should be one of the strategies included in the project. Both the state department of education and the school district had identified increased family involvement as a goal in the school improvement plan as well. The strategies developed emphasized helping families collaborate with teachers to ensure their children’s academic success. A specific goal of increasing family help with homework assignments was established, because this has been identified as a major component of effective family involvement (Cotton, 1995). In addition, issues of cultural sensitivity and increased cultural competency were addressed within this topic.

Participants. The study included three types of participants: schools, special education teachers, and students served in special education programs. A middle school and a high school from a suburban/semi-rural county volunteered to participate in the study. The middle school had 1,249 students, and the high school had 1,400 students. The two schools had similar student demographic characteristics, with approximately 22% of the students at each school from racial or ethnic (non-Caucasian) backgrounds. Approximately 33% of the students in both schools qualified for free or reduced lunch, sometimes used as a socioeconomic indicator of poverty in a school.

This study focused on special education teachers of students in these disability categories: (a) specific learning disabled (SLD) students who spent 50% or more of their school day in special education classes (n = 263); (b) emotionally disturbed (ED) students (n = 56); and (c) educable, mentally handicapped (EMH) students (n = 23). The special education teachers at the middle school (n = 11) and the high school (n = 7) who taught these students volunteered to participate in the collaboration with the researchers to develop and implement the intervention strategies in their classrooms.

Level of Implementation. A scale was developed to measure implementation of the ESMs; this scale was modeled after the intervention validity checklist (IVC) system (Vaughn, Hughes, Schumm, & Klingner, 1998). The IVC scale included a description of the evidence needed in order to determine that a teacher in a project school had implemented an ESM strategy with fidelity as described in the manual. To assist in interpreting this assessment, the same scale was used to assess teachers from comparison schools who had not received training. The results of the fidelity of implementation revealed wide variation among the strategies. The area with the highest degree of implementation was positive behavioral supports, with teachers implementing 76% of these strategies. Parental involvement had the next highest level of implementation, with teachers implementing 67% of these strategies. The implementation level for formative evaluation was 62%, and for reading, 51%. Teachers at comparison schools who had not been involved in the project were also assessed to determine the degree to which they implemented the project strategies. In almost all cases, the project teachers were implementing more of the strategies than the comparison teachers, suggesting that the project teachers were using more evidence-based practices than typical teachers in the district. Teachers participating in the project exhibited higher implementation rates for the formative assessment, family involvement, and positive behavioral support manuals, but for reading their implementation rates were similar to those of the comparison teachers. Based on self-report, teachers in the project were more likely than comparison teachers to measure the current academic functioning level of their students. In the area of formative assessment, project teachers reported using rubrics to delineate expected knowledge for students and used charts and graphs of students’ progress more often than the comparison teachers. Project teachers were more likely to indicate having systematic communication with parents and reported using all aspects of positive behavioral supports at rates higher than those reported by comparison teachers.

Student Outcomes. To determine whether students in special education benefited from the evidence-based strategies used by their teachers, the five outcome areas were assessed for 87 students (SLD: n = 57; ED: n = 13; and EMH: n = 17) over 18 months. These included attendance, academic achievement (in reading and math), level of inclusion (e.g., amount of time a student was exposed to the general education curriculum), rates of suspensions, and discipline referrals. School records were reviewed for number of absences, discipline referrals, and out-of-school suspensions for each student. The class schedules of the students were also analyzed to determine the amount of time spent with nondisabled peers experiencing the general education curriculum. The Wide Range Achievement Test III (Wilkinson, 1993) was used to measure academic skill levels in reading and math. These measures were obtained at three points in time: at the beginning (Time 1) and end (Time 2) of one school year and at the beginning (Time 3) of the next school year.

There were no significant changes over time in either math achievement or number of absences. For students in special education settings due to emotional and behavioral disabilities, there was a significant reduction in the rate of office referrals, F(2, 11) = 8.50, p < .01, and out-of-school suspensions, F(2, 11) = 6.40, p < .01. Additionally, all students increased their level of reading achievement over time, F(2,83) = 5.16, p < .01, and increased their time in general education settings, F(2,83) = 22.87, p < .01. Figures 1, 2, 3, and 4 portray changes in the office referrals, out-of-school suspensions, reading achievement, and time spent in general education settings. Figure 1 illustrates a significant decline in the number of office referrals received by students identified as Severe Emotional Disturbance/Emotional Handicap (SED/EH) over time. At Time 3, the students in the SED/EH groups were receiving office referrals at rates similar to students in the Educable Mentally Retarded (EMR) and SLD categories. The number of office referrals received by students in the EMR and SLD categories did not significantly change over time.

Figure 2 shows an increase from Time 1 to 2 in the number of days spent in out-of-school suspension for the students in the SED/EH category and a decrease from Time 2 to 3. Students in the EMR and SLD groups did not significantly change over time. Students in the SED/EH group spent significantly fewer days in out-of-school suspensions during the first half of the second school year (Time 3) than they did during the first half of the first school year (Time 1).

Figure 3 illustrates changes in WRAT reading scores for the entire sample. Reading scores increased over the first school year (Time 1 to 2) and dropped slightly over the summer break. However, scores at Time 3 were higher than scores at Time 1.

Figure 4 displays the increase in the time students spent in general education settings with their nondisabled peers. The entire sample increased from Time 1 to 2 and experienced a larger increase from Time 2 to 3.

The initial results of this project are encouraging and supportive of continued efforts in increasing evidence-based strategies to improve outcomes for youth with disabilities. The partners, including researchers, teachers, administrators, and family members, reached consensus about these strategies. This article describes the process used to achieve the collaborative partnership, the identification of evidence-based strategies, the development of associated manuals to support implementation by special educations teachers, and the results of subsequent evaluation. Despite of a solid partnership with support from all members, the process of implementing new strategies was found to be a formidable task for teachers, requiring the development of an effective infrastructure to support the achievement of innovative practice. Further efforts to refine processes and materials have been initiated, with the goal of supporting expanded use of effective strategies by all special education teachers.

Carr, E. G., Dunlap, G., Horner, R. H., Koegel, R. L., Turnbull, A. P., Sailor, W., et al. (2002). Positive behavior support: Evolution of an applied science. Journal of Positive Behavior Interventions, 4(1), 4–16, 20.

Cook, B. G., & Schirmer, B. R. (2003). What is special about special education? Introduction to the special series. The Journal of Special Education, 37, 139.

Cotton, K. (1995). Effective schooling practices: A research synthesis 1995 update. Portland, OR: Northwest Regional Educational Laboratory.

Duchnowski, A. J., Kutash, K., Sheffield, S., & Vaughn, B. (in press). Increasing the use of evidence-based strategies by special education teachers: A collaborative approach. Manuscript submitted for publication.

Forness, S. R., Kavale, K. A., Blum, I. M., & Lloyd, J. W. (1997). Mega-analysis of meta-analyses. Teaching Exceptional Children, 29, 4–7.

Gersten, R., Vaughn, S. R., & Schiller, E. P. (2000). Contemporary special education research: Syntheses of the knowledge base on critical instructional issues. Mahwah, NJ: Lawrence Erlbaum Associates.

Greenwood, C. R. (2001). Bridging the gap between research and practice in special education: Issues and implications for teacher preparation. Teacher Education and Special Education, 24(4), 273–275.

Greenwood, C. R., & Abbott, M. (2001). The research-to-practice gap in special education. Teacher Education and Special Education, 24(4), 276–289.

Landrum, T. J., Tankersley, M., & Kauffman, J. M. (2003).What is special about special education for students with emotional or behavioral disorders? The Journal of Special Education, 37, 148–156.

President’s Commission on Excellence in Special Education. (2002). A new era: Revitalizing special education for children and their families. Washington, DC: Author.

Santa, C. M., Havens, L. T., & Maycumber, E. M. (1996). Project CRISS: Creating independence through student-owned strategies. Dubuque, IA: Kendall/Hunt.

Swanson, H. L. (2000). What instruction works for students with learning disabilities? Summarizing the results from a meta-analysis of intervention studies. In R. Gersten, S. R. Vaughn, & E. P. Shiller (Eds.), Contemporary special education research: Syntheses of the knowledge base on critical instructional issues (pp. 1–30). Mahwah, NJ: Lawrence Erlbaum Associates.

Vaughn, S., Hughes, M. T., Schumm, J. S., & Klingner, J. (1998). A collaborative effort to enhance reading and writing instruction in inclusion classrooms. Learning Disability Quarterly, 21, 57–74.

Wilkinson, G. S. (1993). The Wide Range Achievement test (3rd ed.). Wilmington, DE: Wide Range.

Forness, S. R., & Kavale, K. A. (1994). Method analysis research: Methods and implications. In J. Rothman & E. J. Thomas (Eds.), Intervention research: Effective methods for professional practice (pp. 117–131). Chicago: Haworth Press.

Heward, W. L. (2002). Ten faulty notions about teaching and learning that hinder the effectiveness of special education. Journal of Special Education, 36(4), 186–205.

Reid, R., Gonzalaz, J. E., Nordness, P. D., Trout, A., & Epstein, M. H. (2004). A meta-analysis of the academic status of students with emotional/behavioral disturbance. Journal of Special Education, 38(3), 130–143.

Research and Training Center for Children's Mental Health, University of South Florida. (2005). Real manuals for real teachers (Increasing family involvement, Reading strategies, Providing academic feedback, Positive behavior support). Retrieved December 6, 2006, from http://cfs.fmhi.usf.edu/policy/rmrt/

Slavin, R. E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15–21.