News	The Results Aren't In by Buck Quigley

Last week I wrote a news brief about ResulTech, the Maryland-based company that is up for a $1.7 million service contract extension from the Buffalo Schools to “continue on-going technical support at Academy School 44.” A school board vote was scheduled for last Wednesday to approve the contract that had already been composed, approved by legal counsel, signed by the district and the vendor, and notarized.

But instead of giving the contract immediate approval last Wednesday, the board moved to give it closer scrutiny. Only a day earlier, Buffalo Teacher’s Federation President Phil Rumore had circulated a harsh critique that cited widespread disapproval among teachers for the ResulTech program at the school for troubled students.

In my story, I mentioned that when I called ResulTech’s headquarters, the phone rang 30 times and no one answered. When he read that, it prompted a call from David (Pat) Wright, ResulTech’s president of education services. He left a message for me at Artvoice—we have something called voicemail—and I called him back.

I explained that I’d left a message at the school for a member of the ResulTech staff named Marla, as well as school principal Gregory Mott, before trying the ResulTech headquarters. No one called me back while I was writing the story. But now Wright was anxious to explain to me why the company has no answering machine.

“Well, the phone system’s being replaced there, but that’s beside the point,” he said. “I just regret the fact that a more thorough job of researching the allegations could not have been made before the article was written—but I guess that’s journalism today.”

He has a point. There were a number of things I could have included if I’d had more time to do research, and if people had been more cooperative beforehand, and if ResulTech had sprung for an answering machine.

One thing I would have pointed out is how ResulTech purchased the old K-Mart store at 998 Broadway for $1.25 million dollars on May 11, 2007. The historical address was the location of the flagship Sattler’s store that was torn down in the late 1980s. The K-Mart closed in 2002, and the place has now been vacant for six years despite its location directly across the street from the Broadway Market.

In the accounts of the transaction that were published in Business First last year, Wright seemed optimistic about the future of that piece of real estate. “Putting a school there is just one option. It’s close to the medical corridor. There’s an excellent transportation network. It is eligible for Empire Zone credits,” he said then.

When I spoke with Wright last Friday, he explained that the initial plan was to convert the old K-Mart and lease it back to the city as the new site for Academy School—similar to the way Emerson Academy is leased, he said. He was quick to point out this was purely speculative on ResulTech’s part—no promise had been made by the Buffalo schools.

But according to an April 27, 2007 Buffalo News article, there was strong neighborhood opposition to the idea even before the deed was signed. Concern was voiced by activist Samuel A. Herbert, who said: “The problem we foresee is the Broadway Market becoming the new hangout for a whole new group of people who have no respect for authority.” Wright countered that buses would bring students to the back of the building. The front of the building would have meeting spaces the public could use, and the gigantic parking lot would become a park.

By May 25 of last year, Wright, quoted again in Business First, sounded more like a real estate developer than an educator. “Frankly, there are other avenues besides education,” he said. It might make a good distribution center, or a health-care facility.”

Wright gave no indication when I spoke to him last Friday that the property is, in fact, scheduled for auction on July 31, to be “sold on site regardless of price.” (That’s what an astute photojournalist like AV’s Rose Mattrey, who had a little time to take pictures and notice a huge sign mounted on the wall, can learn.)

So apparently ResulTech is going to take a bit of a hit on its $1.25 million investment of one year ago. When I called Wright to check on this as we were going to press, he said he had no knowledge that the building was going to be auctioned.

In the past two years, the school district has paid ResulTech a total of $5.4 million to provide technical support for roughly 500 students. ResulTech leased that many laptop computers the first year, but only half that many students came to the school. The same thing happened the second year.

That’s why this year’s contract is worth a million dollars less—because it’s been scaled back to serve 300 students. As it is, Resultech’s program appears to cost the schools over $10,000 per student per year. Which is, of course, in addition to regular school costs like heat, electricity, furniture, lunches, and, well, teachers.

Wright admits that today, two years into the program, most of the teachers at Academy School 44 have not been fully trained in the use of the ResulTech model. Incredibly, according to two audits of the program, that appears to be no hindrance to the program’s success.

The new contract extension states that the program has been evaluated twice. Once the district contracted with the National Dropout Prevention Center/Network for a report that favorably judged the ResulTech model. The other evaluation was completed on May 18, 2007 (one week after ResulTech signed the K-Mart deal) by the University at Buffalo. The university received approximately $4,000 to produce an eight-page evaluation, according to the report’s author, Dr. Mansoor A. F. Kazi at the Program Evaluation Center at the UB School of Social Work.

Kazi worked with ResulTech, the school principal, and the community superintendent on the report with no teacher input. Here’s an excerpt from this retroactive evaluation of the school’s first academic year:

We were given a list of 283 students, and of these 61% were males. The main research design was a single-system design for each student as well as a pretest posttest design for 206 students at the school for whom data on marking periods and/or Autoskills information was available. 70 students who were described as ‘left’ were not included, and another 7 students were not included due to missing data. We also used a comparison group design in relation to those who had received the Autoskills component and those that had not yet experienced this part of the ResulTech blended curriculum at the school. To enhance the rigor of the evaluation design, we used a new approach in evaluation drawn from epidemiology, namely the testing of binary logistic regression models to determine what interventions work and in what circumstances (Kazi, 2003). First, the sample was divided into two groups, e.g. the students that improved on each of the school outcomes (e.g. their grades) and those who did not. Then the variables that may be influencing the outcome (the components of the intervention applied to date) were entered in a forward-conditional model. In this method, the variables that are actually influencing the outcome are retained in the equation, and those that are also significant within the model provide an exponential beta which is interpreted as an odds ratio, indicating the odds of the intervention achieving the outcome where the significant factor(s) may or may not be present (Jaccard and Dodge, 2004). This evaluation has also helped to identify some shortcomings in the way data is recorded at both the school and the district levels, and paves the way for more robust prospective studies in the future.

Translation: The findings are a scientific interpretation of incomplete data. Nevertheless, they were enough to move the program ahead into the second year. Kazi’s 2007 executive summary recommends that a real-time evaluation of the program’s progress “should be carried out at the end of each marking period in order to investigate the circumstances in which the ResulTech blended curriculum is more or less likely to be effective, and to inform the future implementation with the aim of helping more students to be successful.”

Kazi has not heard from the Buffalo Public Schools since he finished the initial report a year ago. So although the evaluation criteria of the current contract extension states that “two external evaluations were conducted by the University at Buffalo and the National Drop-out Prevention Center in 2006-2007 and evaluation will continue through 2008-09 by both entities based on the criteria listed under expected outcomes,” there has thus far been no assessment of the program’s second year.

According to Wright, one thing they did learn from that first year was that letting the students bring the laptops home was a bad idea. The practice, a big draw at the start of the program, was discontinued in the second year.

What is probably most troubling—and least surprising—about the whole situation is the way the district excludes the input of its teachers in the studies it commissions. Rumore’s survey asks teachers if the ResulTech program is aligned with New York State standards helps students meet those standards. Of the 14 replies, nine strongly disagree, four disagree, while only one agrees. Among the comments: “To follow their program means to knowingly set kids up to fail.”

If the school board approves the extension of the ResulTech contract, it will add up to a total of $7.1 million spent on technical support for about 700 students over the course of three years. There are four ResulTech staff at Academy School, two of whom are part-time.

By comparison, $7.1 million could have hired over 70 new teachers and paid them all the starting salary of $32,897 over the course of the past three years.