One of our primary goals is to increase the number of people in state, local, and federal government who understand how to evaluate research and use these tools to implement and advocate for evidence-based policies."
Katherine Michelmore and Natasha Pilkauskas
Ford School students take on a variety of different roles after graduation—some do research, some will engage in advocacy, others will become civil servants. As instructors of two of the core research methods courses that our students take, we have several goals that we want our students to take away from their time at the Ford School. One of our primary goals is to increase the number of people in state, local, and federal government who understand how to evaluate research and use these tools to implement and advocate for evidence-based policies.
Why do we need to evaluate whether policies work?
First, resources are limited. We need rigorous, credible evidence to ensure that resources are not wasted. Understanding which policies are most effective helps ensure that dollars are well-spent. It is easy to find research that supports a particular opinion; we teach our students to evaluate the limitations of different research methods to determine whether the research is credible. Sometimes an observational study will produce one set of findings, but another methodological approach, like a randomized control trial, might find different results. For instance, correlational studies show that individuals who participate in workplace wellness programs go to the gym more often and report being in better health. But when researchers randomly assigned access to these programs, the methodological approach often referred to as the “gold standard” for evaluating policies, the correlational evidence falls flat—wellness programs do little to improve employee health.
Second, policies do not always have the intended effects. We teach students to not only evaluate research that conforms with the expected impacts, but to also consider research that might show negative effects or unintended consequences. For instance, the “ban the box” policy, where employers stopped requiring job applicants to check the box indicating that they have been convicted of a felony, was intended to reduce discrimination against individuals with criminal records. However, rigorous evidence has shown that the policy had the unintended consequence of increasing discrimination against young Black applicants because employers used other characteristics to infer potential criminal records.
Third, rigorous evidence-based research can highlight tradeoffs we face in investing in policies. Sometimes policies have both positive and negative impacts. For example, in our research on the 2021 expansion of the Child Tax Credit, we find that the credit reduced food and housing insecurity. Yet other high-quality research also suggests that the credit had some small negative impacts on employment for some parents but also positive employment effects for others. We teach students that policymakers have to weigh the pros and cons of potential consequences in making policy decisions and to use the best evidence available to make these choices.
Policy evaluation research is challenging to do well. It can be hard to perform ethical experiments in policymaking with human behavior, which can be unpredictable. Our goal as public policy professors is to educate our students on the benefits and limitations of different methodological approaches so that they can make better policy decisions, no matter what role they find themselves in after leaving the Ford School.
By Katherine Michelmore and Natasha Pilkauskas, associate professors
More in State & Hill
Below, find the full, formatted spring 2024 edition of State & Hill. Click here to return to the spring 2024 S&H homepage.
More news from the Ford School