Simplifying test questions
Do students with learning problems benefit?
Readers of SET—whether they are administrators, teachers, parents, or others interested in special education—are likely to have heard about the idea that content-area tests in science, math, and other areas discriminate against against students with disabilities, including especially those who do not read well. Are the questions on tests of science achievement, for example, written so that they are especially difficult for students with learning disabilities?
Nadine Cruz Neri and Jan Retelsdorf, of the University of Hamburg, tested this (and related) questions in a study published OnLineFirst in Exceptional Children May 2022: "Do Students with Specific Learning Disorders With Impairments in Reading Benefit From Linguistic Simplification of Test Items in Science?" The paper is slated to appear in the forthcoming issue of the printed version of EC, to be published in October of 2022.
Here is the abstract:
Previous research illustrated that reading comprehension and science performance correlate highly. Because students with specific learning disorders with impairments in reading (SLD-IR) show deficits in reading comprehension, they may struggle to perform in science. As language in science is characterized by linguistic complexity, the question arises whether students with SLD-IR can be supported by reducing linguistic complexity. The aim of this preregistered study was to investigate whether students with SLD-IR benefit more from linguistic simplification in science than their peers without SLD-IR. The sample consisted of 70 students (age, M = 12.67; 50% female) with n = 35 having SLD-IR. Applying a multilevel logistic regression model, we found neither a main effect of linguistic simplification nor an interaction effect (differential boost) on science performance. However, students with SLD-IR performed significantly lower in science. Implications include further investigation on how to support students with SLD-IR in their science performance.
Professors Cruz Neri and Ratelsdorf employed exemplary procedures in planning and executing their study. SET readers who conduct research themselves will recognize the strengths in their pre-registration of the study as well as their reporting of the data and materials from the study. For those readers who are not familiar with these “open science” research methods, just understand that the methods promote trust that the researchers took steps to ensure transparency, that their research is not harboring hidden (whether simply mistaken or outright fraudulent) misinformation.
Now, to be sure, the results of this study will need to be replicated (another important feature of open science), but the idea is that this study can likely be trusted.
The take-away is that re-writing test questions to make them less difficult for students who, despite their reading disabilities, have the knowledge and skills to answer the questions correctly, may not be helpful.
We educators may need to bark up another tree...uhm, maybe teaching our students more effectively?
Source
Cruz Neri, N., & Retelsdorf, J. (2022). Do students with specific learning disorders with impairments in reading benefit from linguistic simplification of test items in science? Exceptional Children, 89(1), 22-40. https://doi.org/10.1177%2F00144029221094049
Thanks, John, for sharing the results of this study that's being published by Exceptional Children. I suspect special educators around the world (and not just in the US and Germany) have wondered about this type of test accommodation or modification. Not all would know about the results of this study or think to look at the OnLine first option of Exceptional Children to find this first answer (subject to replication, as you noted). What you've done is share this important information to SET's 450 readers around the world, and that's a great example of what SET can and does do. Keep it up!
I applaud Neri & Retelsdorf for their finely executed study. The results are interesting, believable, but counter-intuitive. F
First, consider the authors’ premise: science learning and reading ability are correlated. There should be no claim that making the reading — i.e., the text — “simpler” will cause science learning to improve. The underlying reading difficulties relate more to underlying cognitive abilities.
It is amply demonstrated that young or learning disabled students can respond more accurately when asked to discriminate responses as opposed to produce a correct response. And, if acceptable, linguistically simpler responses are more likely to be accurate than producing more cognitively challenging language.
For example,
“Why is the ocean warm?”
is more cognitively effortful to process and formulate responses to than “Does the sun make the ocean warm or cool? I would argue that both questions use the same vocabulary and, although the linguistic structures vary, it is the response burden on the learner that is of greater consequence.
Reading difficulties may contribute to but are not the same as understanding and learning from text.
Second, teachers, and adults in general, automatically simplify their spoken language when interacting with babies, tots, and others thought to be linguistically limited. “Baby talk,” for example, occurs universally independent of language and orthographies.
What the pervasiveness of this linguistic accommodation offers to teach us that there are complex but common information-processing principles of interpersonal communication operating that may include but not be wholly represented by characteristics of text (e. g., grammar and vocabulary).
What is missing — not in this study— one study can only do so much — is a good, empirically-based understanding of what “simplification” means as a component of instruction. And not only what is it that simplifies learning in response demands of lessons, but also what principles of “simplification” should teachers and curriculum designers be encouraged to use in a chain of instructional episodes that will lead to a better performance in more passive but independent reading of text.