I think that's an important part of the reason that it's important to employ objective scientific methods to ascertain to effects (beneficial or detrimental) of instructional methods, procedures, and practices—a point with which I suspect you agree. That's why I'm so reluctant to accept someone's statements (e.g., "she is so much better since we started the XYZ program" or "those kids have improved so much!") as convincing, compelling evidence of effectiveness.
I'm *not* arguing that people's perceptions of improvement are unimportant. I'm just saying that just saying (that's not a typo) some students "seem better" can't be equated with those students actually having better outcomes. We need strong measures to make informed evaluations and decisions.
In their forth-coming paper for Exceptional Children, "Evidence-Based Assessment in Special Education Research: Advancing the Use of Evidence in Assessment Tools and Empirical Processes," Betsy Talbot, Andres De Los Reyes, Devin M. Kearns, Jeannette Mancilla-Martinez, and Mo Wang present a strong case for letting the sun shine on important measures in special education. It is still available for free for just a little longer at https://doi.org/10.1177/00144029231171092 (and I hope CEC and Sage will keep it open access for a lot longer).
Just a reminder that people of high intelligence and great talent can be fooled.
So true, Jim. Thanks for saying it.
I think that's an important part of the reason that it's important to employ objective scientific methods to ascertain to effects (beneficial or detrimental) of instructional methods, procedures, and practices—a point with which I suspect you agree. That's why I'm so reluctant to accept someone's statements (e.g., "she is so much better since we started the XYZ program" or "those kids have improved so much!") as convincing, compelling evidence of effectiveness.
I'm *not* arguing that people's perceptions of improvement are unimportant. I'm just saying that just saying (that's not a typo) some students "seem better" can't be equated with those students actually having better outcomes. We need strong measures to make informed evaluations and decisions.
In their forth-coming paper for Exceptional Children, "Evidence-Based Assessment in Special Education Research: Advancing the Use of Evidence in Assessment Tools and Empirical Processes," Betsy Talbot, Andres De Los Reyes, Devin M. Kearns, Jeannette Mancilla-Martinez, and Mo Wang present a strong case for letting the sun shine on important measures in special education. It is still available for free for just a little longer at https://doi.org/10.1177/00144029231171092 (and I hope CEC and Sage will keep it open access for a lot longer).