Education Secretary Spellings to pow-wow with...Jon Stewart

Posted by helen at 12:05 PM

With the real news slowing to a winter-molasses trickle, Secretary of Education Margaret Spellings takes time out of her busy schedule to appear today on Jon Stewart's Daily Show. We wonder when New York's own doubles act, Chancellor Joel Klein and Rev. Al Sharpton, will take their Education Equality Hour from web radio to the tv studio.

For staunch statisticians (and civic-minded parents), Eduwonkette digs deep into math "progress" today -- with ELA on deck for tomorrow. Short take: Gaps in race-based scores, which we asked about here and here, could persist for decades, long after Klein, Sharpton, Spellings et al have folded their big tents and decamped from education leadership.

Bloomberg, Klein to school House panel

Posted by helen at 9:45 AM

Quiet week in NYC? Head down to D.C.: Tomorrow morning, Mayor Bloomberg and Schools Chancellor Joel Klein will address a House panel on progress in urban education, along with D.C. Mayor Adrian M. Fenty, Klein protegee D.C. Schools Chancellor Michelle Rhee and public school leaders from Chicago and Atlanta.

We bet we'll hear about test score gains and closing the achievement gap -- but we doubt the conversation will include troubling nuances, like the fact that race-based gaps between brighter kids widen over time, even as they narrow for kids with lower skills. And we bet we won't hear the nitty-gritty on why level 4 test scores have dropped for middle schoolers this year: Will anyone ask about the price bright students pay in a system so focused on raising low-level student skills?

We'll likely hear about charter schools and merit pay, about leadership pipelines and increasing accountability. We'll hear about rising grad rates -- but bet the numbers they cite will be based on old data, as the newly calibrated scores are yet to be made public.

Will we learn anything new? We doubt it, but we'd love to be wrong. As it stands now, though, our bet is on celebration over substance, and photo-ops over hard questions.

Weekly news round-up: scoring students, scoring Klein, no more summer vacation?

Posted by Lindsey at 12:03 PM

It was the last week of school, and the big story was the generally higher test scores, although the controversy continues over what the scores actually mean. Chancellor Klein was riding high on the test results, although the teachers slammed his performance in a UFT survey. New Orleans superintendent Paul Vallas, said to be short-listed for an eventual successor, says that New York students might say bye-bye to future summer vacations. Large middle schools are the first in line on the chopping block, however, as Klein suggests that he plans to slice and dice them into smaller schools (reported first here, on our blog). Maybe smaller schools will tone down the 8^th grade graduation frenzy. At best, they'll avoid serious issues, like apparent negligence in one Brooklyn junior high school.

Another study confirms what what we already know: there is a woeful lack of playgrounds at New York elementary schools. Let’s hope the new grade school in midtown includes outdoor play space.

The Times ended the school year with a summer storm of local and national school stories: career programs seem to work; a segregated retention program is, unsurprisingly, controversial; a NYC Harbor-based high school builds confidence (see their profile for more); an immigrant parent program boosts involvement; and rent assistance keeps helps families in one place, and kids from switching schools. Whew.

The Times also cautions: summer means bad nutrition. Keep healthy and cool!

Test score bounce: Looking at the numbers

Posted by helen at 8:28 AM

New York City and State's big gains in test scores lead the news at the Times and Daily News, and are featured at the Post and the Sun, which focuses on charter-school progress. But amid the celebratory, double-digit party (and leaving aside, for the moment, critical questions about score inflation and comparisons with national tests), disturbing trends persist, and -- not surprisingly -- get far less play than testing's great leap forward.

Have a look at the test score "deck" from the DOE's Department of Assessment and Accountability, which breaks out scores by grade and race.

The achievement gap that yawns between white and Asian students and their black and Hispanic peers has narrowed, but continues far too wide: Overall, 80% of white students earned level 3/4 (grade-level and higher) on the ELA, compared with 54% of black students and 53% of Hispanic kids. That's a 26% or 27% gap. Even if it closes at the rate of 2 or 3 points a year (the recent, upward trend), that's 9 or 12 years, or many kids' entire public-school career, before the races achieve parity -- if white and Asian kids' scores don't rise, which they likely will (again, tracking Bloomberg-era trends).

The abyss that separates 8th grade's middling progress from 4th grade's high scores is even more threatening: About two-thirds of white eighth-graders, 65%, earned levels 3/4 on the ELA; just over one-third of black and Hispanic students (36% and 33%, respectively) posted similar scores. Taken together, 43% of the city's eighth graders scored level 3/4 -- which means that nearly six in ten will proceed to high-school officially reading below grade level.

Cue the party horns here (or not).

Update: According to DOE, the scores were embargoed on State directive, meant for school use in planning placements (as if year-round testing didn't yield sufficient data) and available to parents on request, but not publicly released until their presentation to the Regents yesterday.

A toast to test scores

Posted by Lindsey at 5:15 PM

It was a love-fest today at PS 178 for New York City’s educational leadership. Ongoing battles over budget cuts were tabled (momentarily) as the Mayor, Chancellor Joel Klein, UFT president Randi Weingarten and CSA rep Ernie Logan lauded city children’s performance on the state ELA and Math tests, which were announced today, and are posted on line here. The credit for the steadily rising test scores (with still-glaring gaps between grade 4 and grade 8 achievement), was generously shared as speakers thanked each other, the children, the parents, the teachers and, of course, themselves.

Each speaker in turn emphasized how much work educational reformers in New York City have yet to accomplish.

“It’s a wonderful day for New York,” the Mayor said, before adding this caveat: “If history looks back and says, 'this is a high point,' shame on us.”

Using kids' test scores, DOE conducting secretive experiments on teachers

Posted by Philissa at 9:52 AM

Who knew I was already right when I hypothesized two weeks ago that the DOE was hoping to change the way teachers are evaluated? Well, besides Eduwonkette, who left a comment telling me so, and at least 140 principals whose teachers are already being judged according to their students' test scores in an initiative so top secret that even the teachers don't know about it? Very few people, it appears, according to the New York Times.

In the already-underway experiment, which the Times was the first to report, the test score gains of students at 140 schools will be used to judge their teachers' success. The DOE is setting "predicted gains" for teachers based on their students' skills, experiences, and backgrounds — and then crunching the numbers to see if the teachers meet those goals. The DOE told the Times, which broke the story, that it doesn't plan to use the results to make hiring or firing decisions about individual teachers. But Chris Cerf, who apparently has been deputized to talk up the program, said the results could be one factor used in those decisions, and that ultimately making the results public (a la the progress reports) would reward good teachers and put pressure on bad ones. Certainly, the DOE must be interested in providing more ammunition for the teacher firing squads assembled earlier this year.

Naturally, the UFT's Randi Weingarten, who has backed down in her opposition to other controversial plans, including the Teacher Performance Unit, sounds angry about this one, telling the Times that she and the city disagree on whether results from this pilot or its expansion could be used under the teachers' contract to make hiring or firing decisions. (On the other hand, the Times says the UFT has known about the experiment for four months, but we haven't heard any complaints until now.)

The initiative also appears to undercut the little agency afforded teachers in determining how performance pay is distributed this year. I'm pretty sure that we don't know how many of the schools included in the performance pay pilot elected to distribute their earnings across the whole faculty rather than to individual teachers, but I think it's safe to guess that's what happened in most schools. Now the DOE is doing the divisive, problematic work its teachers declined to do.

The Times predicts a battle this summer between the DOE and the UFT over the experiment results. Let's hope Randi Weingarten (or, potentially, her successor) is up for the fight. The DOE is abusing test score data, which aren't meant for this kind of crunching, and keeping teachers in the dark about how they're being evaluated. Regardless of the quality of the research (though even that is questionable — Eduwonkette wonders whether the experiment is ethical given that many of the research subjects don't know they are part of an experiment at all), the way the DOE has gone about this one is just not right.

After evaluating students, principals, and schools, test scores to rate teachers, too

Posted by Philissa at 11:52 AM

Since the DOE has demonstrated that it will do whatever it wants, a good way to predict future DOE initiatives is to pay attention to what DOE officials say ought to be done. So when DOE bigwig Christopher Cerf participates on a panel about the "dismal" state of teacher evaluations and decries teachers' "deep antipathy" to being evaluated in a meaningful way, we can assume that somewhere inside Tweed, someone is thinking about new ways to rate teachers. Unfortunately for those of us who think the influence of test scores should be limited, Cerf also said he is "unapologetic that test scores must be a central component of evaluation,"
Education Week reports from the panel.

In fact, Cerf said at the panel that DOE leaders are working on an evaluation system that will look at how far teachers raise their students' test scores. As I recall, one of the papers presented at the Research Partnership conference in October drew on data that showed how far individual students progressed within each classroom, so evidently the bones for such a system must already exist. I imagine the larger obstacle for the DOE will be getting the UFT to agree to use a new evaluation system that relies on hard data instead of observation by other teachers. Of course, the UFT hasn't been much of an impediment to any of the DOE's other initiatives, even when those initiatives appeared not to be in the best interest of teachers.

State's accountability system has bad news for city schools

Posted by Philissa at 9:08 AM

The state has released its own list of elementary and middle schools in good standing and in need of improvement under No Child Left Behind — and the news isn't great for the city or its progress reports.

The state removed 18 city elementary and middle schools from the list but added 64, bringing the total number of city schools not in good standing to 318. Many schools that received D's and F's on their progress reports are considered in good standing with the state, including at least two of the schools that the DOE has announced it will close this year. And many other schools that received A's and B's made the state's list of failing schools.

City education officials say there is "correlation" between the two lists because as a school's progress report score gets higher, it is more likely to be considered in good standing by the state. Still, the discrepancy between the two lists makes sense; after all, the two accountability systems focus on different things. No Child Left Behind looks only at the percentage of students scoring at proficiency each year, while the progress reports look at individual student improvement over the course of each year. The higher number of failing schools this year on the state's list could have to do with more students being tested, as the Post suggests, or on the fact that the state's requirements are getting stricter each year as we get closer to 2014, when No Child Left Behind expects every child to be proficient on state tests.

More on the NAEP scores

Posted by Philissa at 11:43 AM

I bet Joel Klein was thankful for the Thanksgiving-induced reprieve from criticism. When the NAEP scores came out last week, the DOE touted them as proof of success. But more reality-driven commentators were quick to note that in all but one category -- 4th grade math -- scores were flat and that even in that category, the gains are only there if one makes the comparison using numbers that predate the Bloomberg-Klein reforms. The Queens Chronicle even called the NAEP scores "a damning blow to the Bloomberg administration."

Then came the news, broken by the Sun's Elizabeth Green, that New York gave more students testing accommodations than any other city in the country. With a quarter of students taking the 4th grade math test and about 20 percent of students taking the other tests receiving special accommodations, some testing experts think the results aren't worth much. "When you change the statistics for 25% of the people who are guaranteed to be at the lower end, that's going to have a tremendous impact," an NYU professor told the Sun.

A couple of weeks ago, NYC Public School Parents wondered whether the report card fiasco would prove to be an "emperor's new clothes moment" for Klein and Bloomberg. The NAEP scores have done much more to show that the DOE's reforms unfortunately have not produced the improvements the DOE says they have. And the more Joel Klein insists that everyone else is wrong and he is right about how to interpret test scores -- as he did in an email, sent last week to all DOE employees, decrying the unfavorable press coverage -- the sillier the DOE looks.

NAEP results out, city kids either improving or stagnant

Posted by Philissa at 4:21 PM

Scores on the National Assessment of Education Progress tests are out, and New York City kids did great! Or not. Let's see what folks have to say about the results of the test considered "the only nationally representative and continuing assessment of what America's students know and can do in various subject areas."

"City Students Stalled on National Education Tests," from the Sun: "Scores on a math test for fourth-graders went up, but others are statistically flat since 2005."

"New York City Public School Students Make Gains on 2007 NAEP tests," a DOE press release: "New York City students made impressive gains on the 2007 National Assessment of Educational Progress (NAEP) tests, with particularly significant progress achieved by 4^th graders in mathematics compared to their peers in other cities and by Black 4^th-grade students in both reading and math."

Who's right? You be the judge. Check out the scores in math and English for yourself.

Independent research board to audit DOE data

Posted by Philissa at 12:51 PM

Following up on its revelation that increasing test scores might have more to do with easier tests than smarter kids, the Daily News now reports that Chancellor Klein is setting up an independent audit bureau to review test data, which historically have been reported by the same city and state education departments being judged by the tests.

The independent Research Partnership for New York City Schools has been in development for the last year, according to its website, which lists members of the bureau's working groups but hasn't been updated in months. I'm pleased to see the DOE taking seriously criticisms that cut to the core of its recent reforms, but Sol Stern of the conservative Manhattan Institute is wise to question whether the bureau's members will be able to evaluate the data impartially, given that many of them have "an interest in what the research will show."

The bureau, which is being supported with private money, will hold a conference Oct. 5 but is likely to take the entire school year to get fully up and running, the Daily News reports.

Are test scores really improving?

Posted by Philissa at 10:55 PM

The chancellor touts rising test scores as proof that his reforms are working just about every chance he gets. We've already seen historian Diane Ravitch tear down these claims. Now, a new report in the Daily News suggests that an increase in average test scores is correlated with an increase in how easy the tests are. That sounds obvious, until you remember that test makers are not supposed to make tests easier or harder from year to year because test results must allow apples-to-apples comparisons of student and school achievement.

The Daily News also conducted its own experiment to show that the 4th-grade math test in 2005, a mayoral election year when students posted record gains, was 7 percent easier than in 2002. The numbers in the Daily News articles are a bit daunting, but as always it's useful to remember that numbers don't always tell the truth and that politics, not just student performance, can factor into test results.

Study: NYC charter schools outperform public schools

Posted by Unknown at 9:59 AM

A new study on New York charter schools provides compelling evidence that charter school students see greater test score gains than they would have in regular public schools. The study, written by Harvard Economics Professor Caroline Hoxby and the National Bureau of Economic Research's Sonali Murarka, can be downloaded here. (Be forewarned: the study is 81 pages long, but the Executive Summary outlines the main findings in two pages.)

Hoxby and Murarka find that charter schools raise student performance by about .09 standard deviations in math and about .4 standard deviations in reading. We can put those numbers in terms of our standardized tests: On New York's standardized tests, students receive scores from 1 (not meeting standards) to 4 (exceeding standards); a score of 3 is considered proficient. Charter schools raise math scores about 12% of the distance between a score of 1 and a score of 2, or between 2 and 3, etc. For reading scores the gain is about3.5% of a performance level. Keep in mind that these gains are in addition to whatever gains the students would have made in ordinary public schools.

The findings in Hoxby and Murarka's study are particularly convincing because they use truly random sampling to compare charter school students and regular students. The notorious challenge in evaluating any educational policy is that it is often extremely difficult to separate the effects of the policy from other correlations. For example, if we tried to compare the effects of private school and public school, we certainly couldn't just compare standardized test scores of public school students with those of private school students, since the students differ in many ways besides the school they attended. If private school scores were 20% higher than public school scores, for instance, some of that difference might be a result of the differences in education quality, but some of it may come from differences in parental education or socioeconomic status. Even small class size, which many parents and educators have long believed to be good for student achievement, has been very difficult to measure how beneficial small classes are, since students often differ in other ways as well. (If parents who tend to be more concerned for their children's education tend to demand small classes more often than other students, and if those concerned parents also read to their kids more, for instance, some of the test score differences between small-class and large-class students may come from differences in how much they're read to at home.)

So how can we accurately measure the effects of some educational policy? If students are randomly assigned to either "treatment groups" (those with the effect) and "control groups" (those without), we can compare those two groups without worrying about other factors that might bias the results. Fortunately, New York City charter schools provide an ideal opportunity for such a comparison. In New York, when more students apply for a charter school than can be admitted, they are selected by lottery-- a random sample! So Hoxby and Murarka compared the scores of students who were selected for admission with those who entered the lottery but were not selected. This type of comparison is typically considered so convincing that researchers will weigh the results of one good random-comparison study more heavily than any number of other, non-random results.

Finally, the study notes that charter school advocates would say the true benefits of charter schools are even greater than the findings in this study suggest. Since one of the supposed benefits of charter schools is that they bring competition for regular schools, advocates would say that charter schools actually improve the quality of education in regular public schools, as those schools are forced to improve to keep students from leaving for charter schools. If that's true, even the non-charter school students, i.e. the "control group," would have benefited from the effects of charter schools.

Klein's statistics painting too pretty a picture?

Posted by Unknown at 3:16 PM

Chancellor Klein's PR staff deserves as much credit for the City's recent statistical gains as the administration's reorganization, according to Sol Stern's recent column in City Journal. Stern is a scholar at the conservative-tending Manhattan Institute. Stern's piece gives yet another look at the history of NYC's education reform, but it goes beyond a simple history to do a bit of investigative journalism, delving into the tactics of Klein's formidable public relations staff. Stern writes:

The most notorious case of Bloomberg’s data manipulation occurred during the 2005 mayoral race. In May of that year, city hall bused education reporters to P.S. 33, a poor, predominantly minority school in the Bronx, where Bloomberg congratulated the children, their teachers, and Principal Elba Lopez on a miracle: 83 percent of the school’s fourth-graders scored at grade level on the 2005 reading test, compared with only 35.8 percent the previous year—an unheard-of one-year gain of close to 50 percentage points. The school’s score was just 4 percentage points below the average for the state’s richest suburban districts. Further, the mayor announced, the percentage of the city’s fourth-graders passing the state’s reading test had risen by a “record-breaking” 10 points in just one year.

If Bloomberg had really introduced accountability into the city’s education system, the implausible P.S. 33 scores would have raised red flags at the education department and perhaps even prompted a fraud referral to the city’s Special Commissioner of Investigations. Instead, the mayor got the political boost that he sought, with front-page headlines hailing the “historic” gains. Almost no commentary pointed out that fourth-grade reading scores rose by almost 10 percentage points in the rest of the state, too, suggesting that the 2005 test might have been easier than the previous year’s.

Though the full article is nearly 4000 words, it's well worth the time. Thanks to NYC Public School Parents for bringing this one to our attention.

Klein: ELL students will wait longer before taking tests

Posted by Unknown at 10:02 AM

Yesterday NY1 reported on Chancellor Klein's new plan to allow English Language Learner (ELL) students more time before requiring them to take standardized tests. Whereas ELL students currently have to take the tests during their first year in school (even students who haven't yet been in the States for a full year), Klein plans to change that requirement, exempting those students from required tests for their first two years.

While I think Klein is probably correct to exempt ELL students from state tests, since those students hardly need to spend time taking exams that won't yield meaningful results, I can't help but be skeptical about Klein's motivation for the change. ELL test scores are, of course, far below the citywide average, especially the scores of ELL students during their first two years of NYC public education. Therefore removing them from the test pool will probably result in a significant jump in test scores, giving the mistaken impression that test scores have greatly improved. With the end of Bloomberg's administration in site, and mayoral control of the city's public schools sunsetting in 2009, I can't help but think this measure could be a last push to show test gains under the Bloomberg-Klein reforms.

Of course, I could be wrong-- one could avoid the misleading test results by removing from past averages the scores of first- and second-year ELL students when calculating improvement statistics. Then the statistics would truly track how the same types of students were performing from year to year-- comparing apples and apples-- which is the only way to accurately measure the effects of school reform over time. We'll see if the DOE takes this step, or if instead they forgo statistical accuracy for the sake of political gain and claim credit for a test gain that never occurred.

More on math scores (updated)

Posted by Unknown at 12:32 PM

6/17 update: For a summary of the reasons to be wary of the new math test scores, check out Diane Ravitch's article over at the NYC Public School Parents blog. Ravitch urges readers to "wait patiently to see whether the recent gains on the state tests are reflected on the national tests when the results are posted in November 2007."

6/15 post: The math scores published last week have attracted a wave of commentary, everything from ecstasy to serious skepticism. Today a few more publications weigh in. Elizabeth Green at the New York Sun reports that city officials are touting charter school scores as evidence that charter schools are working. She writes:

This year, 74% of city charter students scored proficient on the state math test, up from 66% last year, a review of state data by a procharter group, the New York City Center for Charter School Excellence, found. Just 65% of students citywide scored as well this year, up from 57% last year.

Meanwhile, the Times published a cautiously optimistic editorial, concluding that "all signs suggest that the city and state are on the right track."

Math scores are out; NYC students are doing better

Posted by Philissa at 4:02 PM

The state released scores from the 2007 math exams today, and scores for New York City students seem to have jumped. About 65 percent of students in grades 3-8 scored at or above grade level on the state test, up from about 57 percent last year.

The New York Times reports that the city's scores were better than those of almost every other large school district and that they are closer than ever to the state average. Statewide, about 73 percent of students scored at or above grade level on the tests, up from 68 percent last year. The state's press release notes that scores for middle school students and students with disabilities jumped the most, and that the gap between scores for black and Hispanic students and white students narrowed somewhat. The Times suggests that the rise in test scores could be a result of students' growing familiarity with the test format and content, which changed last year, and teachers' familiarity with new state math standards.

Of course, what looks like a straightforward jump in test scores can sometimes be more complex when the numbers actually get crunched, as historian Diane Ravitch recently pointed out when she took a close look at this year's reading scores. So we're looking forward to seeing the numbers in more detail. For now, we're cautiously optimistic about the scores and proud of the city's students.