In an era of shrinking public education budgets, school districts cannot afford to make the wrong decision when they hire a teacher or cut a program. To make sure they reach the right answers, administrators are turning to Annenberg Professor Brian A. Jacob, whose data analysis has helped guide urban public school reforms across the country.
"Districts really are becoming much more interested in data and analysis than they were 10 or even five years ago," Jacob said. "Increased availability of electronic information has made a lot more analysis possible."
Jacob is actively adding to that wealth of analysis. With colleagues from Columbia University he's developing more reliable ways of evaluating teaching candidates for Washington, DC's public schools.
"One solution [for cash-strapped districts] is to be better about hiring in the first place," said Jacob, who is conducting the project pro bono. "In order to do that you have to understand whether there is anything that can predict who will be an effective teacher."
Rather than relying on resumes, Jacob employed methods "that usually you wouldn't use in a job interview," including written essays, personality inventories, and classroom auditions, with candidates teaching a 30-minute lesson plan to current students. Performance reviews of the nearly 500 teachers hired under the trial selection system are now under way, with the goal of helping DC recalibrate its hiring process in the future.
Jacob has seen the devastating setbacks of teacher malpractice. In 2011, more than 170 teachers and principals across 44 Atlanta public schools were accused of correcting students' answers on standardized tests to meet performance benchmarks. In what is believed to be the largest investigation into teacher cheating, Jacob is serving as an expert witness, working with district officials and attorneys to determine whether student "erasure patterns" are sufficiently compelling to move forward with dismissal hearings or criminal charges. Many of the implicated staff have admitted wrongdoing and no longer work in the district.
"Every student is going to have an occasional answer erased and changed from wrong to right," Jacob said. "But if you have a sufficiently large number of these cases, it starts to look suspicious."
Jacob's ability to provide impartial feedback is an asset to groups like Excellent Schools Detroit (ESD), a publicprivate partnership to improve the city's education system. Jacob has been advising the ESD on its efforts to collect and analyze student outcome and school performance data.
With widespread skepticism of state and city officials amongst Detroiters, Jacob hopes the Ford School's Education Policy Initiative–which he co-directs with Professor Susan M. Dynarski–can become a trusted source about school performance and student outcomes for parents and policymakers.
"I think we have a pretty good reputation as honest academic brokers," Jacob said. "We're going to analyze the data and present what we find in a transparent way."
With widespread poverty, frequent turnover of administrators (by election and appointment), and thousands of teachers who must be receptive to reforms and faithful to their implementation, Detroit faces many of the challenges unique to urban districts.
"How do you move an organization of that scale?" asks Jacob, who noted Chicago has 30,000 teachers citywide. "Implementing any reforms with consistency and fidelity is difficult."
Adding to the challenge is what Jacob calls "healthy tension" between districts and researchers.
"District officials are often looking for answers very quickly about very specific program components," said Jacob, who also mentors a group of fledgling policy analysts who are designing performance benchmarks for Philadelphia's schools. "There's often this pushback when I try to explain, 'If we really want to answer the question, we can't do it in three weeks.'"
It appears Jacob and his proteges have no shortage of work ahead of them.
Below is a formatted version of this article from State & Hill, the magazine of the Ford School. View the entire Spring 2013 State & Hill here.