Handcuffing Education
 

Gerald Coles
 

In January of this year, George W. Bush began his second year in office by signing educational legislation titled "No Child Left Behind Act." He emphasized that it embodied his greatest concern, his "signature issue," the one that also defined his tenure as governor of Texas and his domestic policies as president. To ensure that all children get off to a good educational start, he emphasized at the signing ceremony, a no-nonsense approach to reading, the foundation of all education, was necessary:
 

We're going to spend more money, more resources, but they'll be directed at methods that work, not feel-good methods, not sound-good methods, but methods that actually work, particularly when it comes to reading. So this bill focuses on reading. It sets a grand goal for the children. Our children will be reading by the third grade. And so, therefore, we tripled the amount of federal funding for scientifically based early reading programs. We've got money in there to make sure teachers know how to teach what works. We've got money in there to help promote proven methods of instruction. We know what works.
 

"Scientifically-based reading instruction" appears nearly 50 times in the legislation and refers to instruction that proceeds step-wise from smaller to larger units of written language: from phonemic sounds (smallest unit of speech sounds), to letter-sound relationships (i.e., phonics), to word identification, to reading fluency, and eventually to meaningful comprehension. Instruction is direct, explicit, and systematic, conveniently packaged in reading programs that include textbooks and an array of instructional materials, and manufactured by big publishing companies. Teachers, serving as middle-managers and, adhering to the pre-planned lesson sequence in a teacher's manual, lead their students lesson after lesson, skill after skill, day after day. Testing that is heavily skills-based is an ever present goal and shaper of this instruction.
 

Ironically--and distressingly--there is no scientific evidence supporting either the effectiveness of this approach or the need for mandating it through national legislation requiring school systems to adhere to the "scientific" imperatives in order to obtain funds. Further, there is no evidence that it has worked in Texas, where its achievements have been touted, and where the claims helped establish Bush as a "compassionate conservative" who cares about children, especially poor children.
 

Gi-Go
 

The chief evidence cited in support of Bush's "scientific" reading instruction is the Report of the National Reading Panel (NRP), a document published several months before Bush became president, and frequently referred to both by Bush and his Secretary of Education Rod Paige (www.nationalreadingpanel.org). Although the congressional charge that created the panel asked for an assessment of the "effectiveness of various approaches to teaching children to read," since the majority of NRP panelists are champions of the aforementioned model of reading development, the report merely justifies that partiality and does not offer an objective appraisal of "various approaches."
 

The chief devices used by the NRP to evaluate the studies it reviewed were effect-size calculations and a meta-analysis. The former estimates the extent that something has an effect--large, medium, small, or absent--on something else, in this case, the effect of aspects of instruction on achievement. A meta-analysis does the same with a group of studies. These devices can provide useful information, but they also have substantial deficiencies, such as the well-known "Gi-Go" problem of "garbage-in-garbage-out," a term referring to the inability of a statistical analysis to exceed the quality of a study or group of studies it uses.
 

Reexamining the report's effect-size calculations and meta-analysis on their own statistical ground is important for appraising its validity. However, the report's unsubstantiated conclusions can only be partly understood this way because of deeper problems that fully undermine the statistical analyses and, in turn, the Bush legislative mandates. The following is one example.
 

Correlation vs. Causation
 

The NRP concluded that phonemic awareness training is the best predictor of students' later reading success. (Phonemic awareness refers to the ability to separate and manipulate speech sounds mentally and orally, as when blending or separating phonemes in order to identify words.) The report then describes one study as supporting this and the additional conclusion that "letter knowledge" (knowing the names of letters) is the second best predictor of reading success. The report's description suggests that these predictors are "causes" of reading achievement.
 

However, the accuracy of this conclusion becomes questionable when one includes the study's third strongest predictor of reading achievement, success on a "finger localization" test--a predictor not mentioned in the report. This test is conducted by having a child's vision blocked while he or she identifies which of her or his fingers has been touched. Despite its predictive correlation with future reading achievement, finger localization skill could not be considered "causal" to learning to read, and no educator would suggest finger location training as a beginning reading method.
 

Zip codes are also good "predictors" of academic achievement. A student's zip code (an indication of family income and education, quality of schools in the area, a child's access to educational experiences, etc.) is strongly correlated with future school success. However, this correlation does not make zip codes a cause of academic achievement. Unfortunately, given the failure of some reading researchers to understand the difference between correlation and causation, it is possible that one of them might concoct an experiment to see if the reading achievement of children in a poor, urban area with terrible schools might improve if those children were given the zip code of children in an affluent, suburban area with excellent schools.
 

The NRP Report also fails to mention that the researchers who did this study, which the NRP used to justify its own conclusions, offered an explicit caveat about confusing correlation with causation. Yes, they did find that letter knowledge was a strong predictor of future reading, but they emphasized its predictive strength did not mean that beginning readers needed to know letter names in order to get off to a fast, secure start in reading. Although "knowledge of letter names has been traditionally considered the single best predictor of reading achievement, there appears to be no evidence that letter-name knowledge facilitates reading acquisition," the researchers stated.
 

Letter-name knowledge is likely to represent experience with early written language that contributes to literacy attainment. That knowledge--like the knowledge of various skills, such as phonemic awareness--can be considered to be a "marker" of these experiences and accomplishments. None of this complexity and these distinctions are captured in the report's simplistic summary.
 

"Reading" and Comprehension
 

A major problem in the report is the ever-changing definition of the term "reading." In most of the studies the report used, the term refers to reading real or nonsense words (e.g., nef, pim), or reading aloud smoothly and rapidly. Seldom does it mean comprehending text, a process most people would define as key in "reading." Furthermore, there is an amazing lack of evidence that the skills-emphasis instruction touted as "scientifically-based" promotes comprehension.
 

The meta-analysis used ten studies, from which twenty statistical relationships were derived, to evaluate the effects of phonemic skills training on reading comprehension. Among the twenty, six supposedly showed a "large" effect, six showed a moderate effect, and eight showed either a small or a negative effect.
 

In my forthcoming book, Reading Unmentionables: Damaging Reading Education While Seeming to Fix It (Heinemann), I review every one of the studies on comprehension used in the report. Here, because of length constraints, I will review only the six "large" effects, the strongest evidence claimed in the report. Three of these came from a single study, a revealing fact not mentioned in the report and obtainable only through careful review of numerous details in one appendix.
 

This study divided poor readers into four groups, two of which received training in sound categorization (for example, learning which words shared beginning, middle, and ending sounds.) In addition to this training, one of these two groups used plastic letters to help learn the sounds. The third group learned how to classify words and picture cards into categories (for example, hen and bat are animals). All three of these groups had the training added to their regular classroom instruction. The fourth group had classroom instruction only. At the end of the two-year training period, the two groups that received training in sound categorization had scores in reading and spelling that were statistically superior to those of the other groups, and the group that had also used plastic letters scored better than the one that had not.
 

Thus the "large" effects on reading comprehension that the NRP alludes to are derived from comparisons of instruction versus no instruction or of instruction versus categorizing words and pictures. The study does show that specific training in phonemic awareness, with and without learning letter/sound relationships, can help children earn better reading and spelling scores than if they do not receive this training. However, it shows nothing about comparisons of effective teaching and comprehension.
 

With respect to the charge from Congress, we may ask, what are the "various approaches" included in this study, since no group was engaged in meaningful instructional alternatives? Moreover, while the previous study I discussed offered some information about classroom reading instruction, the two publications about this study offer none. Without this information, we can draw no conclusions about the classroom instruction and the need for this additional training. We see again how the simplistic model of the meta-analysis not only distorts the study's findings but also terminates thoughtful inquiries about teaching and learning at the very junctures where they should begin.
 

The other studies containing "large" effects on reading comprehension can be briefly summarized. One, conducted in Spain, found that, after pitting a skills-training group against a group that colored and cut and played various games, the skills training had a greater impact on reading.
 

And the regular classroom curriculum? The original paper says only that reading instruction "began with a global approach, using familiar and simple utterances followed by a more analytical phase, where they focused on phonemes." Little additional information is provided. But for the NRP no more was needed.
 

The final study that yielded two of the NRP's "large" effects was a study in Finland, where "formal reading and spelling instruction starts the year the child reaches seven years of age." While one group of kindergartners was in a phonemic skills training program for the entire kindergarten year, the children in the control group received no alternative written language instruction. Nevertheless, the results of this something-against-nothing investigation provided the fifth "large" effect on reading comprehension as far as the report is concerned.
 

The sixth and final "large" effect is even less useful for the meta-analysis because the regular classroom instruction in this Finnish study used a "phonics program," a fact the report fails to mention. Consequently, we have a study in which training in phonics skills is appended to phonics classroom instruction. I suppose we can conclude here that children taught with a phonics reading program will benefit from an additional phonics training program, but this is hardly evidence against other methods of teaching.
 

The Texas "Miracle"
 

If the evidence for the "scientific" reading instruction mandated in the Bush reading legislation is not in the research literature, perhaps it is in his so-called "Texas educational miracle." According to a "Bush for President" news release, "Reading performance in Texas has improved: 88 percent of the third-grade students passed the reading portion of the state assessment test in 1999--up from 76 percent in 1994." Minority students too were described as having "made strong gains" on the state's own Texas Assessment of Academic Skills (TAAS). For example, "African-American students in both the eighth and fourth grade increased their passing rate on the reading exam by 23 points between 1994 and 1999."
 

Texas Myth
 

Comparing the TAAS scores with those of Texas students' performance on the National Assessment of Educational Achievement, educational researcher Walt Haney concluded that the "Texas Miracle" was really the "Texas Myth" because the seeming huge achievement gains demonstrated on the TAAS were not duplicated on this non-partisan measure (www.olam.ed.asu.edu/epaa/v8n41/). Fourth- and eighth-graders overall did about the same as students nationally--not worse, but not better. For example, in Texas grade 4 reading achievement had a mean of 217, exactly the same as the national average on the reading test at 4th grade.
 

For minority students, however, contrary to claims about the racial achievement gap narrowing in Texas, outcomes were worse. Between 1992 and 1998, the NAEP reading scores for fourth-graders showed an increase in the gap between white and minority students.
 

Why have students done better on the Texas Assessment of Academic Skills than on national tests? Interviews with teachers, Haney reported, suggest that teaching to the test was a large part of the answer: "In most schools TAAS practice quizzes were administered on a regular basis with emphasis on teaching to the TAAS format, such as having students practice 'bubbling' in answers on machine scorable answer sheets." Similarly, classes have emphasized teaching TAAS-related content while deemphasizing content not related to the test.
 

Texas Education Facts
 

The Bush "signature issue" is, in fact, a simplistic--and wholly inadequate--boot strap "solution" to educating children, especially poor children. Instead of focusing on a well-trained professionally competent teaching force and on providing them with all that is required to implement a rich education for children, "scientific reading education" offers seemingly "teacher-proof" packaged programs. The Bush approach offers education-on-the-cheap while enhancing the profits of publishers like McGraw-Hill, the major producer of the packaged programs.
 

To take one example of Bush's indifference to building a professional, experienced teaching force, under his governorship Texas teacher salaries ranked 36th among states. One study found that 28 percent of Texas teachers had to take second jobs. Although Bush frequently mentions giving the teachers a $3,000 pay raise in 1999, the largest in fifteen years, he leaves out the fact that he fought for a $1,500 raise but lost to the Democrats, who had first proposed a $6,000 pay raise.
 

Bush's "signature issue" also masks his colossal indifference to children. While proclaiming a dedication to ensuring that children obtain basic reading skills--a far cry from ensuring a full education--there was no equal concern for children's living conditions and overall quality of their lives that have an incalculable effect on learning success or failure. For example, Texas ranked second among states in the percentage of people--especially children--who went hungry, and third in the percentage of malnourished citizens. After vetoing a bill to coordinate hunger programs in Texas, reporters asked Bush about hunger in the state. "Where?" was his reply.
 

Under Bush, Texas was tied for the third highest percentage of children in poverty. Just three states provided lower welfare help. Under Bush, Texas slashed its food stamp payments, an essential program for the poorest of children, by $1 billion. Also under Bush, Texas ranked second in the percentage of poor children who lacked health insurance. In 1999, when Texas was flush with a budget surplus, Bush initially fought to block some 250,000 children from receiving affordable health care. While fighting this insurance program, he declared a legislative emergency to push through a $45 million tax break for oil well owners, saying "People are hurting out there." He also had the opportunity to invest more of the surplus in education, but instead pursued tax cuts.
 

All of this while posing in one classroom after another, reading a storybook, and thereby exhibiting his devotion to the "Texas miracle."
 

President Bush
 

Bush carried his Texas game plan into the White House. Accolades for his educational budget and dedication to education were ceaseless, with few commentators bothering to investigate beyond the façade.
 

What exactly was Bush's budgetary great leap forward? Had he gotten his way, his increase in the education budget of approximately $2.5 billion for fiscal 2002 would have been about 5.9 percent. In contrast, the fiscal 2001 education budget, the last Clinton budget, was $42.1 billion, an increase of 18 percent over the previous year's funding. The final budget that Bush signed, following revisions in the House and Senate, was $48.9 billion, approximately a 16.2 percent--not an insignificant increase, but less than the last Clinton increase of 18 percent (the 48.9 billion is also about the same size as the increase Bush recently asked for the military budget).
 

Although the budget puts $900 million into promoting "scientifically-based reading education," other areas of education were cut and more will be cut in the FY 2003 budget. Support for libraries has decreased, even in light of the documented substantial inequalities between classroom libraries in rich and poor school systems, and despite strong documentation showing direct association between access to print and reading achievement. The cuts, an American Library Association executive calculated, could "buy nearly 1.1 million hardcover books," and the executive added, "We really thought if they were serious about education, they would give us somewhere close to the resources that are needed."
 

During the Budget Committee's deliberations, the Democrats offered several amendments that would have provided additional funds critical for improving education, such as class-size reduction, and school renovation and construction. The Republicans defeated all these amendments.
 

In the FY 2003 budget, Bush has proposed cutting all funds for rural education, school counseling, and educational technology. Frozen are funds for teacher quality programs. Barely enlarged are funds for Head Start, a 1.9 percent proposed increase that would allow the program simply to maintain its current enrollment level of under 50 percent of all children eligible for the program.
 

Little attention is given to school conditions or to ensuring a professionally solid teaching force (e.g., in Los Angeles, overcrowding has forced children to attend school on a staggered year-round schedule; half of all new teachers, and a quarter of all teachers, lack formal teaching credentials).
 

At the same time, as organizations like the Children's Defense Fund document, approximately 12.1 million children--one in six--live in poverty. Although there has been a recent drop in the poverty rate, children are more likely to be poor today than they were 20 or 30 years ago. The child poverty rate is highest for African Americans (33 percent) and Latinos (30 percent), but by international standards, it is also exceptionally high for white children (9 percent). For children under six, the "near poverty rate" is 41 percent!
 

Twelve million children are in households unable to afford adequate and nutritious food. About 3.6 million children live in "severely substandard housing." Eleven million children have no health insurance and "are less likely to receive medical and dental care when they need it" or have adequate follow-up care "to manage chronic illnesses like asthma or diabetes."
 

It is within this context that Bush's compassion and concern for children must be judged. Even if his "scientific reading education" did have a scientific basis, focusing on reading education, without concern for these catastrophic realities would have to be seen in a questionable light. Instead, we can only conclude that the Bush reading education legislation, containing deficient solutions and false promises, is part and parcel of the rest of his policy of indifference to all but the wealthy.
 

In several years, after the failure of this educational charade becomes clear, we can be certain that the blame will fall not on national policy or on the scientific reading programs, but on teachers, for not correctly implementing them.
 

Speaking at a middle school while campaigning for president, Bush asserted, "We want our teachers to be trained so they can meet the obligations, their obligations as teachers. We want them to know how to teach the science of reading. In order to make sure there's not this kind of federal--federal cufflink." Whatever he meant, isn't it time for parents and educators who understand the deficiencies of Bush's education policies to roll up their sleeves?
 

--
 

Gerald Coles is an educational psychologist living in Ithaca. This article (copyright 2002, Gerald Coles) is adapted from his forthcoming book, Reading Unmentionables: Damaging Reading Education While Seeming to Fix It (Heinemann). Reference sources for this article are the book. The author can be contacted at: gscoles@yahoo.com