Experts in child development underscore that the health and success of each person depend in important ways on what happens before birth, in other words, on prenatal conditions. It turns out that the same goes for good government. Whether a program or policy is set up for success and for continuous improvement depends a lot on what happens in the drafting process, before a bill becomes law.
To understand why, consider the standard “recipe” for a piece of federal legislation. A bill typically starts with a section on its purpose, followed by a section on the allowed or required activities. Grant programs specify eligibility criteria. At the end of the bill is sometimes an evaluation section, requiring that an assessment of how well the program works be conducted within “x” years of passage of the law. To implement the law, federal agencies set up reporting requirements, often onerous, to document whether funds are being used for authorized activities.
{mosads}Despite all this, little attention is paid to creating feedback loops for decision-makers to learn what is working, what is not, and how to adjust to get better results. Clearly, the requirement for a program evaluation is a good thing, and more bills should have one. Without an evaluation, Congress is spending taxpayer dollars without knowing the return on that investment.
But the problem is that evaluations conducted years after a program has launched are not very useful. They do not provide for program implementation to be mapped out in advance so that the most important unknowns about that program can be answered. In short, the standard recipe for legislation is unlikely to produce the type of insights that can best help federal agencies or Congress continually improve their results and cost effectiveness, meaning fewer lives improved and less bang for the buck for taxpayers.
So how should Congress structure bills to prioritize learning what works and continuous improvement? It needs to stop making the evidence component of legislation an afterthought. Instead, it should structure bills so that they require the implementing agency to do the following.
First, create a learning agenda at the start. That means identifying the research or operational questions that the agency and, importantly, stakeholders think are the most important. Maybe it is whether the program will work as well for rural as for urban participants. Or maybe it is whether a slower or quicker version of a certain intervention, such as a training program, will be more effective. Whatever the questions are, they should be identified before a program gets off the ground.
Second, structure the program, as well as the evaluation, to answer the questions identified in the learning agenda. Clarify from the start what types of data and what kind of infrastructure to collect data will be needed for a useful and credible evaluation. This ensures that program leaders are thinking from the start about what types of data they or their state or local program partners need to collect.
Third, ensure that program resources can flow to data and evaluation needs, including at the state and local levels. Those resources will likely be a small percentage of overall funding, especially if administrative data that is already being collected can be used for evaluations, but this ensures the resources for high quality and accessible data.
The closest we have today to Congress taking these steps is with the creation of what are called tiered evidence grant programs, also known as innovation funds. These competitive grants, such as the Education Innovation and Research Program at the U.S. Department of Education, build a commitment to useful evaluation into the DNA of these programs. They do this by ensuring that projects are structured and adequately funded to answer questions that are important to practitioners and policymakers. There are only a handful of tiered evidence grant programs across the federal government, however, so it is only a start.
All types of programs, whether carried out by federal employees, contractors or grantees, will benefit from legislation that charges and empowers decision-makers to identify important questions and to use data and evaluation to continuously improve. That is why it is time to toss out the old recipe for bill writing. In its place, Congress should ensure that federal agencies are helping programs address and credibly answer priority questions about what works and for whom. That, in turn, would be a recipe for results, strengthening a learning culture in which government continually tests, learns and adapts.
Andrew Feldman is a director in the public sector practice at Grant Thornton. He was previously a visiting fellow at the Brookings Institution and a special adviser on the evidence team at the White House Office of Management and Budget. Kathy Stack served for more than 30 years in the federal government, including 28 years at the White House Office of Management and Budget. She was previously vice president for results driven government at the Laura and John Arnold Foundation.