December 3, 2003
Gains in Houston Schools: How Real Are They?
By DIANA JEAN SCHEMO and FORD FESSENDEN
HOUSTON As a student at Jefferson Davis High here,
Rosa Arevelo seemed the "Texas miracle" in motion. After years of
classroom drills, she passed the high school exam required for
graduation on her first try. A program of college prep courses earned
her the designation "Texas scholar."
At the University of Houston, though, Ms. Arevelo discovered the
distance between what Texas public schools called success and what she
needed to know. Trained to write five-paragraph "persuasive essays" for
the state exam, she was stumped by her first writing assignment. She
failed the college entrance exam in math twice, even with a year of
remedial algebra. At 19, she gave up and went to trade school.
"I had good grades in high school, so I thought I could do well in
college," Ms. Arevelo said. "I thought I was getting a good education.
I was shocked."
In recent years, Texas has trumpeted the academic gains of Ms. Arevelo
and millions more students largely on the basis of a state test, the
Texas Assessment of Academic Skills, or TAAS. As a presidential
candidate, Texas's former governor, George W. Bush, contended that
Texas's methods of holding schools responsible for student performance
had brought huge improvements in passing rates and remarkable strides
in eliminating the gap between white and minority children.
The claims catapulted Houston's superintendent, Rod Paige, to
Washington as education secretary and made Texas a model for the
country. The education law signed by President Bush in January 2002, No
Child Left Behind, gives public schools 12 years to match Houston's
success and bring virtually all children to academic proficiency.
But an examination of the performance of students in Houston by The New
York Times raises serious doubts about the magnitude of those gains.
Scores on a national exam that Houston students took alongside the
Texas exam from 1999 to 2002 showed much smaller gains and falling
scores in high school reading.
Compared with the rest of the country, Houston's gains on the national
exam, the Stanford Achievement Test, were modest. The improvements in
middle and elementary school were a fraction of those depicted by the
Texas test and were similar to those posted on the Stanford test by
students in Los Angeles.
Over all, a comparison of the performance of Houston students who took
the Stanford exam in 2002 and in 1999 showed most did not advance in
relation to their counterparts across the nation. More than half of
them either remained in the same place or lost ground in reading and
math.
"Is it better or worse than what's going on anywhere else?" said Edward
H. Haertel, a professor of education at Stanford University. "On
average it looks like it's not." Stanford University has no
relationship to the test.
In an interview, Dr. Paige defended Texas's system, saying that it had
gradually raised the standards for success over the last 20 years.
"Texas measures far more than minimal skills," he said. "The bar is far
above what other districts use."
But questions about Houston's accomplishments are increasing. In June,
the Texas Education Agency found rampant undercounting of school
dropouts. Houston school officials have also been accused of
overstating how many high school graduates were college bound and of
failing to report violent crimes in schools to state authorities.
The Houston officials strenuously defend the district's record.
Kathryn Sanchez, head of assessment for Houston's schools, said
students were doing well on both the Texas exam and the Stanford test,
given the city's large number of poor and minority students. Ms.
Sanchez said that Houston students had also done well on the National
Assessment of Educational Progress, a federally mandated test widely
referred to as "the nation's report card."
On that test, fourth graders in Houston and New York outdid children in
four other cities in writing, to score at the national average. Fourth
graders in New York and Houston also led children in other cities in
reading, yet fell short of the national average. Of all six cities,
however, Houston excluded the most children with limited English from
taking the national assessment, and some researchers suggest that
removing such students may have helped raise Houston's score.
But in interviews, Houston school officials acknowledge that the
progress in the elementary grades peters out in high school. About
13,600 eighth graders in 1998 dwindled to fewer than 8,000 high school
graduates. Though 88 percent of Houston's student body is black and
Latino, only a few hundred minority students leave high school "college
ready," according to state figures.
Miracle or Mirage?
With its own exam to measure pupil achievement, Texas managed to show
educational progress over the last decade on a scale rarely, if ever,
achieved before. But as the state's paradigm for school accountability
became law for the rest of the nation, the authenticity of Texas's
accomplishments has become a major question in education policy.
The Stanford test provides a useful contrast to the state exam, at
least for Houston. More than 75,000 students in grades 3 through 8 and
grade 10 took the state exam as well as the Stanford test from 1999 to
2002. The Times analyzed performances on these tests, excluding
students in special education, and had educational testing experts
review the results. The data were obtained under the state's open
records act by George Scott, president of the Tax Research Association
of Houston and Harris County, a taxpayers group.
"I don't think there was a miracle," said Robert L. Linn, co-director
of the Center for Research on Evaluation, Standards and Student Testing
at the University of Colorado, who reviewed the calculations. "There
were some good positive results, but not extraordinary results like
TAAS seemed to show."
The modest improvements in Houston have implications for the national
debate. "If you anticipate that you can have the gains shown on TAAS
and that's what No Child Left Behind would be requiring in many states
that's not going to be likely to happen, based on this," Dr. Linn said.
The Times analysis of performance on the Stanford Achievement Test and
the Texas exam shows this:
¶Houston students improved from 1999 to 2002 in most grades, but
at only a fraction of the rate portrayed by the state exam. Using a
widely employed statistical measure that allows different kinds of
tests to be compared called effect size, the gains in the average
scores on the Stanford test were about a third of the average gain in
the TAAS scores.
¶Even students with the poorest skills posted high scores on the
Texas test. In reading, a passing score of 70 on the test was the
equivalent to scores below the 30th percentile in national ranking on
the Stanford test in every grade. In 10th grade, passing the state exam
was equivalent to the fifth percentile in the national ranking.
¶While the Houston gains on the Stanford test in some grades were
large enough to be considered significant in educational testing, the
city was not making much headway when compared with national averages.
Some 57 percent of Houston students who took the math test in 1999 and
2002, and 51 percent of those who took the reading test, saw their
standing relative to children around the country either fall or remain
the same.
¶On the Stanford tests, the average reading scores for Houston
students of all races in grades 9 through 11 have actually dropped
since 1999. By contrast, the reading scores for 10th graders on the
Texas exam the only high school grade in which the state test is given
showed a large gain over the same period.
¶The achievement gap between whites and minorities, which Houston
authorities have argued has nearly disappeared on the Texas exam,
remains huge on the Stanford test. The ranking of the average white
student was 36 points higher than that of the average black student in
1999 and fell slightly, to 34 points, in 2002.
"This says that the progress on TAAS is probably overstated, possibly
by quite a margin," said Daniel Koretz of the Harvard School of
Education, who also reviewed The Times's analysis, "And when all is
said and done, Houston looks average or below average."
Tougher Texas Test
While Texas minority students have made gains on the federal
government's mandated national assessment test of reading and math,
they were already largely ahead of the average scores of minority
students from around the country before the current Texas
accountability system began in 1993.
In Houston, the share of college-bound high school graduates that the
Texas Higher Education Coordinating Board deemed "college ready" fell
to 28.5 percent, or 977 students in 2001, from 33.7 percent, or 1,155
students, in 2000, according to the latest figures available. The board
counts only graduates who seek admission to public institutions of
higher education in Texas, and says another 10 to 15 percent may seek
admission elsewhere.
But many here saw the replacement of the Texas exam last spring with a
tougher exam as the most stinging indictment of the test. On the new
test, the Texas Assessment of Knowledge and Skills, or TAKS, race gaps
widened, and passing rates fell.
Officials here now say that TAAS was only a test of "minimal skills,"
paving the way for ratcheting up standards with a new exam.
Dr. Paige contends that the TAAS and Stanford tests could not be
compared because the Texas test gauges mastery of the Texas curriculum
while the Stanford test measures a more general notion of what children
should know in a given grade.
But education researchers disagreed.
"These two tests ought to be telling the same story, and they're
telling different stories," said Dr. Haertel, of Stanford University.
Dr. Paige also argued that statistical anomalies in the results on the
Texas test made comparisons impossible. But testing experts who
examined those anomalies said that, if anything, they would reduce the
disparities between the two tests.
Watching Children Struggle
In one way or another, Jo Arevelo, Rosa's mother, has watched each of
her children struggle through an educational system that was focused
tightly on producing high test scores on state exams.
Last summer, Ms. Arevelo tutored her youngest daughter, 10-year-old
Angelica, in spelling. Because the state exam does not test spelling,
Angelica's teacher never got to it, Ms. Arevelo said one recent
afternoon.
Earlier that day, her son, Joseph, took the preparatory exam for the
SAT college entrance test, but like many other children that day, he
left the exam in frustration mystified by vocabulary words like
parallelism and euphemism, words he had never encountered in school.
Patricia Anderson, a veteran social studies teacher in Houston, said
she was not surprised. Noticing that her high school students could not
answer questions after reading passages in their textbooks, she began
giving them a vocabulary test at the fourth grade level. Typically, she
said, "They flunk it."
"We're all very very frustrated, because all these great scores are
coming out of the elementary schools, and when they get to high school
it's not happening," Ms. Anderson said. "They do not have the skills
they need."
It was not always like this. Many parents welcomed the accountability
system that the Houston district pioneered in the 1980's and early
1990's. It was a way, they reasoned, to force schools in poor
neighborhoods not to write off their children.
And in some places, it seemed to work, said Rene Barrios, lead
organizer for the Metropolitan Organization, a chapter of a group that
monitors public services. But in many other places, Ms. Barrios said,
the system became the single most important measure of school success
and the test itself, for many teachers, became the curriculum. "The
whole system has been taken over by the test," she said.
Rosa Arevelo, who graduated from Davis High with a B average, tried to
keep pace in college. She made flash cards to help her remember what
she studied. She had never learned how to take notes in high school, so
at her lectures in college, she took down everything the teacher said.
Her textbook looks as if it is filled with neon lights: entire
paragraphs are highlighted in bars of bright pink and yellow. In the
unrelenting array of information, she could not tell what mattered.
"When you get to college," she said, "you're just supposed to know. But
nobody ever taught us."
|