NICHCY Connections...
to Research 102: Adding Up the Evidence

Launched April 9, 2004
Updated December 3, 2007
Approx. 5 pages when printed

Author
Dr. Barbara Smith
Research Analyst, NICHCY

So you want to use research findings when you make decisions affecting children with disabilities---what's the best practice for educating them, for determining their placement, for training their teachers, for designing the systems that will help them flourish and succeed. So you start searching for research that's relevant to the decisions you have to make. You run across terms like single study, group design, experimental design, literature review, synthesis, and---ahhh! here's a high-falutin' one---meta-analysis. What's the difference between these terms, and how much weight can you put on the conclusions the research authors draw?

• Research 101
  (What makes for good research?)
  


• Research 102: Adding Up the Evidence (you're here!)
  (How do you combine the findings of multiple research studies?)
  


• Making Sense of Statistics in Research
  (Don't let stats throw you.)
  


• Weighing Info for Its Worth
  (Is this research well done?)
  
  
• Special Education Research: Where to Start?
  (How to begin finding and applying research.)
  
  
• What Works: Can We Say?
  (Where can I find information on evidence-based practices?)
  
  
• Research-Based Resources on Specific Disabilities
  (A starting place for research-based information on disabilities.)
  

Research Reviews

• The literature review.
  "Lit reviews" abound in educational research. Most journal articles reporting research begin here, summing up our knowledge to date on the topic. So how do lit reviews differ from other types of summaries of research?
  www.utoronto.ca/writing/litrev.html
  


• Lit reviews 2: What they are, why you do 'em, how you do 'em, and tips and tricks.
  http://www.library.cqu.edu.au/tutorials/litreviewpages/index.htm
  


• A Webtutorial on preparing scholarly reviews of the literature.
  www.gwu.edu/~litrev/
  


• Basing decisions on the research: Does the new "way" work? Is it worth it?
  Before changing teaching strategies, you want to have enough evidence that the new method will be an improvement.
  http://bmj.bmjjournals.com/cgi/content/full/319/7211/652?
  ijkey=6dc65fa7ca8312d95390a3957b23b3eb07f37cc9&keytype2=tf_ipsecsha
  


• Research reviews and the problem of interpretive bias.
  Even the results of one or two well-done studies aren't enough to make you confident that it's time for a change. Instead, you need to have a body of evidence to support that change. Usually, review articles are a good source of information. These are narrative articles describing an area of research on a particular question. However, authors may have an interpretation bias: Watch out for it.
  http://bmj.bmjjournals.com/cgi/content/full/326/7404/1453?
  ijkey=c8b957d72ed82ab3af241547b7a3c7a1c9cdd93f&keytype2=tf_ipsecsha
  


• Systematic reviews: Trying to avoid the bias by looking at the whole picture.
  Systematic reviews attempt to avoid bias by covering all studies relevant to a given question, putting those results together, reporting strengths and weaknesses of those studies, and putting together a comprehensive summary.
  
• How is a systematic review of research created?
  www.jr2.ox.ac.uk/bandolier/painres/download/whatis/Syst-review.pdf
  


• More on systematic reviews.
  http://ssrc.tums.ac.ir/SystematicReview/Assets/Pai_NMJI_2004_Systematic_reviews_illustra.pdf
  


• How to read papers that summarize other papers.
  http://bmj.bmjjournals.com/cgi/content/full/315/7109/672

Back to top

• Introduction to meta-analysis from research-based medicine.
  www.jr2.ox.ac.uk/bandolier/painres/download/whatis/Meta-An.pdf
  


• Putting separate results together.
  www.lyonsmorris.com/MetaA/index.htm
  


• Steps involved in conducting a meta-analysis.
  www.lyonsmorris.com/MetaA/appendix1.htm
  


• Meta-analysis as a technique: Pros and cons.
  http://bmj.bmjjournals.com/archive/7119/7119ed.htm
  


• The principles and procedures of meta-analysis.
  http://bmj.bmjjournals.com/archive/7121/7121ed.htm
  
  
• Search the NICHCY Research-to-Practice database to find structured abstracts that explain special education meta-analyses in accessible terms.
  http://research.nichcy.org/search.asp
  

Back to top

• Yeah, what is it?
  http://cem.dur.ac.uk/ebeuk/research/effectsize/ESguide.htm
  


• Why do we need to know anything about the effect size, especially if we find that the differences between groups are statistically significant?
  http://www.cemcentre.org/renderpage.asp?linkID=30325015
  


• Want to calculate an effect size yourself?  Try out this effect size calculator.
  http://www.cemcentre.org/renderpage.asp?linkID=30325017
  

Back to top

• An overview.
  www.edres.org/meta/
  


• Meta-analysis in educational research.
  http://pareonline.net/getvn.asp?v=2&n=8
  
  
• Will it work for these students?
  
  • The case for validity generalization.
    http://pareonline.net/getvn.asp?v=2&n=13
    
  
  
  • The generalizability of research results.
    http://www.socialresearchmethods.net/kb/external.htm
    
    
  • More on generalizing results.
    http://www.gifted.uconn.edu/siegle/research/Samples/externalvalidity.html
    
    
• Did the study measure the right thing?
  
• Operational definitions of concepts.
  http://jan.ucc.nau.edu/~pe/hp602web/HP602VA14.htm
  


• More on defining concepts.
  www.sfu.ca/~richards/Zen/Pages/Chap3.htm

Back to top

• Connection to educational research reviews, syntheses, and meta-analyses relevant to children with disabilities and the systems that exist on their behalf.
  


• Tell us what research topics we should add to this list of Connections! Drop us a line at: nichcy@aed.org

Back to top