As dysfunctional as Congress has become, it does manage to enact some useful bills. One of them, the Evidence-Based Policymaking Act of 2018, was championed by outgoing Republican House Speaker Paul Ryan and Sen. Patty Murray (D-Washington). It promotes data-sharing and policy evaluation throughout the federal government.
As popular as the concept is across the spectrum, however, we should all have realistic expectations about its application. While few question the value of basing public policy on careful study of solid evidence, human nature is not easily eluded.
It’s easy to endorse evidence-based policymaking when you assume it will reinforce your existing beliefs, and to assume that your adversaries are the ones whose policies will fail the evidentiary test. To varying degrees, we are all prone to such cognitive temptations as selection bias (noticing and remembering information that fits our preconceived notions or personal interests) and confirmation bias (viewing favorable evidence in the best possible light and contrary evidence in the worst).
While it would be reassuring to assume otherwise, it’s demonstrably clear that motivated reasoning is rampant throughout the political discourse. That doesn’t mean we can’t have informed debates. And it doesn’t mean minds are never changed by solid evidence presented persuasively. But there is nothing automatic about it, because human beings are not automatons.
Over the years, I’ve spotlighted numerous examples of motivated reasoning in action, of what Manhattan Institute fellow Oren Cass termed “policy-based evidence making.” Although I am politically conservative, I don’t limit my examples to those in which progressives are the transgressors.
For example, conservative policymakers are right to observe that the majority of peer-reviewed academic research shows a statistically significant negative relationship between taxes and economic growth. All other things being equal, economies subjected to higher tax rates tend to grow more slowly than those featuring lower tax rates.
However, conservatives often exaggerate the finding far beyond the preponderance of the evidence. Some assert that tax cuts pay for themselves in the short run by stimulating so much growth that governments collect more revenue at the lower rates than they would have at the higher ones. This is theoretically possible, if the initial tax burden is high enough, but rare. It essentially never happens at the state and local level.
At least issues of tax policy are a bit cloudy and debatable. That’s not the case with regard to one of the most egregious errors made by progressives in North Carolina and other states: defending pay supplements for teachers who obtain graduate degrees.
The evidence on this question is overwhelming: teachers with advanced degrees are not more effective, on average, than teachers with only undergraduate degrees.
Here’s what I mean by overwhelming. By my count, there have been more than 100 peer-reviewed studies published since 1990 testing the relationship between advanced degrees and teacher effectiveness. More than 80 percent found no link. Of the handful of studies finding a positive association, the graduate degree is question was typically in science or math, which represent a very small percentage of the graduate degrees for which teachers have received pay bumps.
If we were all truly committed to allowing evidence to guide our policy decisions, the decision of the North Carolina General Assembly several years ago to eliminate any new supplements for graduate degrees would be widely acclaimed. Instead, Democratic lawmakers, candidates, and activists — not to mention some of the policy’s key beneficiaries, education schools — have continued to demand the policy’s reinstatement.
It is certainly the case that no single study or small group of studies, no matter how well designed, can settle any longstanding dispute. That’s not a realistic model for employing evidence in policymaking. Data do not actually “speak for themselves.” They require interpretation. Few interpreters are free from bias.
The best we can do, I suspect, is to ensure that there is a diversity of interpretation in public policymaking — and to commit to reading all of it, not just results that can be expected to confirm our brilliance.