Some components of reading comprehension don't lend themselves well to scientific study, with the ... [+]
The middle of a global pandemic might seem like an odd time to focus on the limitations of science, the very discipline we’re hoping will save us. But when it comes to human behavior and cognition—and specifically, to reading—it can be misleading to draw conclusions based on scientific studies.
There’s one aspect of reading for which scientific data is indisputably helpful—or would be, if more people understood or accepted it: decoding, or sounding out words. The clear consensus from myriad studies is that systematic instruction in phonics is what many kids need and virtually all can benefit from.
But the other aspect of reading—comprehension—is different. Scientific studies generally try to isolate one variable and test it on a “treatment” group and a “control” group for a period of days or weeks. That works well with something like decoding, which involves a defined set of skills that can be taught directly. Results can be seen pretty quickly.
But comprehension is a far more complex and gradual process involving factors that don’t lend themselves easily to scientific measures of effectiveness—with one exception: metacognitive strategies, techniques that help readers think about whether they’re understanding what they’re reading. Many studies of these strategies, generally lasting no more than six weeks, have shown positive effects.
Other kinds of studies have shown that background knowledge and familiarity with the conventions of written language also have powerful effects on comprehension. But those factors take longer than six weeks to develop. As a result, while their effects have been measured, we don’t have scientific data that shows boosting them has a direct impact on reading comprehension. In other words, we have evidence showing that if a “poor reader” knows a lot about baseball, he’ll do better on a comprehension test than a “good reader” who doesn’t know much about baseball—when the topic is baseball. What we don’t have is evidence showing that if you provide a student who knows nothing about baseball (or some other topic) with a lot of information about it over the course of a few weeks, he’ll then do better on a standardized reading test covering other topics. It’s hard to see why that would happen.
Given the abundance of data on the effectiveness of strategies, educators and reading experts have placed greater weight on comprehension “skills and strategies” than on the other elements of comprehension. That has led to a situation where, aside from time reserved for math, the elementary school day is devoted largely to practicing skills and strategies on disconnected reading passages—with little attention paid to building kids’ knowledge or familiarity with writing conventions. Not only is this regime failing to boost reading test scores, it’s not clear anyone intended this to happen, least of all the scientists who conducted the studies. So how did we get here?
One reason is that educators and others often assume that the research behind comprehension strategies also applies to “skills”—like “finding the main idea” or “making inferences”—which are far more commonly taught than strategies. Those skills also appear to be what kids need to do well on standardized reading tests. That has led to an intense focus on them, especially where reading scores are low.
Some reading experts, like Dr. Timothy Shanahan, argue there’s a clear distinction between skills and strategies—but in fact, there are also confusing similarities. It’s true that each type of comprehension instruction arose at different times and with different objectives. Skills instruction has been popular since at least the 1950s, when it was enshrined in textbooks—anthologies of disconnected passages—called basal readers. The passages would be followed by questions that required students to do things like “find the main idea” or make inferences.
Strategies, in contrast, came out of research in the 1970s on techniques used by “expert” readers. Psychologists found that these readers unconsciously monitored their comprehension and came up with ways to help themselves if it broke down—for example, by pausing periodically to summarize what they’d read. The question was whether inexpert readers could be taught to do the same kinds of things consciously. Some reading authorities expressed doubts, including one—P. David Pearson—who wondered if emphasizing strategies over content ran the risk of turning “relatively simple and intuitively obvious tasks into introspective nightmares.”
But by the 1980s, teachers who had stumbled upon the research were already putting strategy instruction into practice in their elementary classrooms. These teachers, part of the “whole language” movement, had rejected basal readers and their comprehension questions, which they viewed as stultifying. But to them—as to Shanahan—strategies looked different.
The research hadn’t been turned into anything resembling a curriculum, so teachers improvised. One influential literacy guru, Ellin Keene, interpreted the findings to mean that teachers should “model” one strategy at a time, spending weeks on each, and teaching them year after year beginning in kindergarten. No studies endorsed Keene’s approach, although eventually people assumed they did.
But even Keene sometimes had her doubts about the difference between strategies and skills. In her book Mosaic of Thought, she described advising a 5th-grader struggling to understand an encyclopedia entry to use the strategy of “identifying which sentence is most important.” (As far as I know, this strategy isn’t backed by research.) The boy, named Jeremy, points to a sentence that begins with the word “ammunition,” explaining he chose it because he likes guns. But he adds that he doesn’t understand the rest of the sentence, which is about the American Revolution—a topic he doesn’t know much about—and contains terms like “Committee of Safety” and “provincial assembly.” Without explaining the words, Keene advises Jeremy to do what “great readers” do: “listen to that mental voice tell them which words … and which ideas are most important.” Later, she wonders whether that advice was any more helpful than the old, ineffective basal approach of telling students to practice the skill of finding the main idea. Ultimately she decides it was, because it helped Jeremy “know specifically on what he should concentrate.”
That anecdote illustrates one thing skills and strategies have in common: if you don’t have enough knowledge and vocabulary relating to the topic to make at least some sense of the text, neither kind of technique will help you. Let’s say Keene advised Jeremy to use the evidence-backed strategy of asking himself questions about the text as he went along. Without more knowledge of the Revolution, he wouldn’t have been able to answer them.
Why, then, have studies shown that strategy instruction boosts comprehension? Probably because some kids don’t understand that the point of reading isn’t just to decode the words but to make meaning from the text. Strategies clue them into that and—if they have the requisite threshold of background knowledge and familiarity with writing conventions—help them do it.
There are other reasons to doubt the distinction between skills and strategies. Arguably, learning to summarize—an evidence-backed strategy—isn’t that different from the “skill” of learning to find the main idea. Even researchers sometimes blur the categories. I’ve seen studies that refer, for example, to “main idea strategy instruction”—or evaluate the strategy of self-questioning using questions about “main idea.”
Ah, experts like Shanahan say, but the crucial difference is that strategies are intended to be used consciously, while skills are supposed to become automatic with practice. But if expert readers use strategies unconsciously, don’t they eventually become automatic as well? And couldn’t a reader consciously use the “skill” of finding the main idea?
There’s also evidence that strategy instruction may interfere with comprehension—that David Pearson was right about “introspective nightmares,” although he later became a strategy enthusiast. One 2009 study compared strategy-focused instruction with lessons centered on the content of what kids were reading. In the content-focused classroom, teachers asked open-ended questions like “How does all this connect with what we read earlier?” Not only did that lead to richer discussion, it also boosted students’ comprehension and retention of information compared to the kids in the strategy-focused classroom. One of the study’s authors, Margaret McKeown, later said that with strategy instruction, “kids aren’t going directly for the meaning of the text. They’re going through, okay, how do I do a summary? Or how do I predict? They have to go through a kind of routine. And we think that sometimes there could be, for many kids, more focus on what that routine is than what the content of the text itself is.”
With the best of intentions, educators and some reading experts have looked at research that is fine as far as it goes but have drawn conclusions it doesn’t support—and that have led to devastating consequences for many children. That doesn’t mean we should dismiss the evidence on comprehension strategies—or even “skills.” There are ways to preserve the baby while throwing out the harmful bathwater. I’ll turn to that subject in my next post.
