A new study on the impact of “money in politics” has been released by the Roosevelt Institute, and sympathetic outlets are already hailing it as a vindication of those who long argued that political spending directly influences policymaking in government. Despite its headline-grabbing claims, however, the study exaggerates the scale of political spending, overestimates its impact, and is riddled with other logical and methodological flaws. The authors are Thomas Ferguson of the University of Massachusetts, Boston, Paul Jorgensen of the University of Texas Rio Grande Valley, and Jie Chen of the University of Massachusetts, Boston.

This paper specifically tests the influence of “high finance” on alterations to the Dodd-Frank Act, and of telecommunications companies on the passage of network neutrality regulations. (This post will focus on the former, for the sake of brevity). The authors’ hypothesis is that lawmakers watered down banking regulations in direct response to financial incentives from corporate interests in that sector. They attempt to prove this theory by examining data on political spending that they say has been previously missed by media and academia. In so doing, the authors aim to reveal the “true size” of such spending beyond regular campaign contributions.

The authors examined the connection between industry political spending and the likelihood of a Democrat in the House of Representatives “changing their mind” on Dodd-Frank (a seemingly polite way of insinuating corruption). In particular, the study’s focus was on Democrats who voted for Dodd-Frank and then later voted for measures to “weaken” it in five subsequent bills.

Overall, the paper purports to find that:

• Every $100,000 that Democrats received from the finance sector increased the odds of “changing their mind” on Dodd-Frank by 13.9%;
• Representatives who voluntarily retired from Congress after the 2014 elections were 3 times likelier to cast a “pro-bank” vote; and
• Representatives on the House Financial Services Committee were 90% more likely to “side with the banks.”

For all the talk that this study has decisively proven correct the arguments for further restrictions on “money in politics,” the reality falls dramatically short. It severely overestimates the level of political spending and the causal relationship between money spent and votes cast. The study’s biggest problems fall into two basic categories: disqualifying flaws specific to its methodology, and basic misunderstandings about the complexities involved with political giving.

I. Everything Is Bribery

The study spends more than half of its space building a narrative of a conspiratorial system whereby special interests hijack the government, wreck the economy, and make themselves rich in the process. But the litany of overbroad definitions, omitted variables, and mischaracterizations in its actual analytical model undermines its ability to quantitatively prove this hypothesis.

a. “Political Spending”

Perhaps most glaring is the “full-spectrum analysis” of political money. It should be noted that the study never cites final estimations for total spending, calculation methods, or its sources, despite insisting that “factors like political money…can be measured fairly precisely.” Nor does it define terms like “outside spending,” which has a conventional definition that may be less helpful in a paper aiming to upend existing conceptions and calculations of political spending. The authors’ categories and the amounts of estimated money for each area are discussed below:

1. “Payments to Lawyers for Services,” an amount that the authors describe as “substantial” but “unknown.” This category is described mainly as contributions from law firms that “have had major banks as principal clients.” Supposedly, “[p]olitical contributions from members of these firms mostly track the interests of their clients,” and thus should be counted as if they come directly from finance. This is nonsensical on many fronts. Wouldn’t lawyers favor increased regulations so that firms need more legal counsel? Is the main political motivation of finance employees (who make up all industry contributions to candidates and parties) really to serve the interests of their firms’ clients? Do other industries that service or benefit from finance also act as their proxies? Given the central importance of finance to the U.S. economy, almost every business should be implicated in this arrangement by the authors’ logic.
2. “Payments to Political Figures,” which are “not easily quantified,” but may be “many hundreds of millions of dollars.” This category refers to things like speaking fees, book contracts, and consulting fees – basically private sector work that a politician might engage in after leaving office. Finance is certainly not the only industry that participates; after all, aren’t, for example, universities that hire former lawmakers to teach also potentially trying to gain influence? Is there no legitimate reason to pay former politicians to utilize their experiences in and knowledge of government? If there is, how is this category subdivided to account for that?
3. “Foundations and Charitable Grants,” which the authors concede are mostly not political. They estimate $296 billion in total giving in 2006, with “maybe 3-5%” ($8.8 to $14.8 billion) being “broadly political.” This description is rife with questions: What is the definition of “political”? How do these foundations and grants directly influence politicians on specific policy questions? The authors allude to instances of grants supposedly meant to curry favor with lawmakers’ spouses, but do not explain how this is incorporated into their analytic model.
4. “Think Tanks” (specifically in Washington, D.C.), which collectively spent about $411 million in 2005. Like “Foundations and Charitable Grants,” there are a host of questions about how this spending influences specific lawmakers on specific policy questions. Think tanks, after all, are usually nonprofit, nonpartisan organizations that study numerous issues.
5. “Value of Stock Tips, IPOs to Political Figures,” which are “very large in certain periods.” The authors note that “stocks in both finance and telecommunications often did extraordinarily well” in advance of the 2008 crash (compared to what other industries is unclear), and that “[o]ne can conjecture that both sectors were likely involved heavily” in insider trading, “but without going through the mountains of evidence that Congress for a long time made very difficult to access, it is impossible to say for sure.” It is hard to respond to “conjecture” without evidence.
6. “Public Relations Spending,” of which, “[s]ome certainly affects politics.” This section is particularly stunning in its imprecision. Not only is there no estimate of this amount or how much of it affects politics, but suggesting that non-political advertising by an industry can be labeled “political spending” undermines the purpose of categorizing such spending to begin with.

The study also accounts for more traditional measures of political spending (“Lobbying” and “Formal Campaign Spending”) that are more strictly regulated. But, here too, the authors fail to tell us the extent to which these factors are emphasized in their model.

Recall that the above-mentioned categories ostensibly represent “political” spending that can be measured “fairly precisely.” The bafflingly expansive and opaque categories constructed by the authors already disqualify them from being taken seriously as an input in any quantitative model. What the authors describe amounts to nothing more than a list of funding sources that they assume are corrupting politicians, but do not bother to accurately count. In short, it is an editorial dressed up as social science.

b. Omitted Variables

After spending over half the paper preaching about corrupt politicians, what does it actually study? It looks at just five congressional votes from 2013 to 2015 to modify portions of the Dodd-Frank Act. The report then looks at the numbers of House Democrats observed “breaking” with their party to cast a “pro-bank” vote. The total number of Democrats that broke with the party on each vote, respectively, is 36, 3, 17, 32, and 3. This is an overall sample of just 91 yeas or nays, which may include multiple votes by the same member of Congress. From these 91 deviations on just five House votes to amend a single piece of legislation covering one issue area, the study concluded that there are “direct links between political contributions to individual members of Congress” and policy decisions in general.

Even within these scant observations, there seems to be too few control variables to draw a causal link. Besides the primary variables examining political money and “pro-bank” votes, there are variables for ideology, whether a Representative voluntarily left the House after the 2014 elections, and whether he or she was a member of the House Financial Services Committee. Not included are numerous other factors that could affect a lawmaker’s vote, such as district economic performance, pressure from party leadership on each vote, media coverage, public opinion on the issue, and the presence or absence of electoral challengers, among other considerations.

To be sure, some of these factors are hard to quantify (though that did not stop the authors from ambitiously re-defining all “political money”), and the authors dismiss others as “fixed effects.” As they explain:

This tactic [looking at multiple votes from the same Representative over time] makes sorting out competing influences much more tractable. It controls automatically for many sources of possible variation, including the individual representative’s own personality, values, and party affiliation. In effect, the procedure turns the representative into his or her own control. No less helpfully, in the very short run, it is not plausible that many other influences, such as variations in the strength of market propaganda in different districts, or most other institutional factors, including district and constituency characteristics, could change rapidly enough to matter. They can thus be treated as constants or “fixed effects.”

By contrast, factors like political money do change, sometimes dramatically, and they can be measured fairly precisely, as we explained earlier, provided one is willing to do the work. [Emphasis added.]

It is hard to imagine a scenario over the “very short run” (presumably the 3-5-year period between Dodd-Frank’s passage and the five subsequent votes) where political spending can change “dramatically” without other factors also changing (most obviously, public opinion, which regularly shifts with changing political winds). The authors posited that interactions with lobbyists, margin of electoral victory, and even change in ideology could be factors, but only included ideology in their model.

The authors’ rationale is that unchanging constituency composition and party-line attitudes about banking regulation meant that there was little reason for a Representative who supported Dodd-Frank in 2010 to independently “change his or her mind” about the legislation 3-5 years’ down the road. However, aspects of Dodd-Frank could have become less popular within a district or had a negative impact on the district’s economy or financial sector that lawmakers observed only after it was implemented. Parties also emphasize some votes more than others – Democratic leaders could have pushed for Dodd-Frank’s passage as a priority to their Party and President Obama, but were less concerned with stopping all subsequent bills that altered specific parts of the law while keeping it intact. Furthermore, a lawmaker could reasonably vote “yes” on Dodd-Frank, thinking it to be a positive bill overall, and still vote “yes” on an amendment that removed a particular provision of the massive and complex bill that she thought was bad for the country or her district.

Had the authors included votes on other banking or economic regulations in their analysis, it may have shed light on whether it was the defectors’ original votes on Dodd-Frank or subsequent votes that were aberrations. Alas, they only provide the vote on Dodd-Frank as a benchmark of lawmakers’ “true” views on financial regulation in general. Any vote contradicting the original vote on passage of Dodd-Frank is assumed to be evidence of corruption.

Given the insufficient variables, not to mention the questionable and vague measurements of political money, it is difficult to put much stock in the model’s results. The study further fails to report any measure of fit – as is typically included in regression analyses – so it is difficult to even fully measure the adequacy of the limited model that the authors employ.

Finally, it should also be noted that one of the study’s topline results – that members of the House Financial Services Committee are 90% more likely to cast a “pro-bank” decision in the five votes examined – is not statistically significant. (Significance levels are usually set at 0.05, meaning a 95% confidence interval. The p-value for this variable is over two times too large to qualify for that level). This is a fact the authors choose to share with their readers in footnote 58 of Appendix I on page 38 of the study. This, to say the least, greatly undermines the authors’ decision to highlight that result anyway – let alone as a topline finding.

II. The Mistakes That Just Won’t Go Away

Past studies that try to find a direct, causal link between political spending and policy decisions by lawmakers have been regularly stymied by one major flaw. Namely, the link is neither direct nor causal. While its Executive Summary might indicate otherwise, the details of this study reveal that it falls into the same familiar trap.

One of the overarching reasons for this is the authors’ reliance on examining the spending of an entire industry. In this study, the culprit is “an almost monolithic financial sector” opposed to Dodd-Frank. Odds are, when you directly refer to an entire industry as “monolithic,” your arguments will lack nuance.

Firms within an industry may have overlapping policy interests, but that does not make them identical. The study itself notes the division between telecom companies supporting and opposing net neutrality, but does not similarly examine differences within the finance industry. For example, Dodd-Frank affects community banks differently than financial behemoths that can better absorb the cost of the law’s regulations, even though they belong to the same industry. To suggest that banks large and small have the exact same views on every bill related to Dodd-Frank is overly simplistic.

Treating firms as unitary political actors can also be wrongheaded – they are made up of executives, employees, and shareholders with varying viewpoints that change over time. Businesses cannot even make direct contributions to federal candidates; at best, they can set up and operate political action committees (PACs) that solicit funds, and whose contributions to candidates are capped at $5,000 per election. In short, contributions from finance firms to candidates actually come from their employees.

If it is simplistic to conflate the interests and goals of firms, it is downright ridiculous to do the same for the various employees of a multitude of businesses. Individuals have distinct reasons for participating in politics, and it is hard to believe that politicians regularly examine every donor’s employer and attempt to ascertain those firms’ policy preferences before casting a vote. They would have to weigh every contribution from every firm and sector, and then decide which money is ultimately the most compelling. This complex calculus presents a difficult balancing act. If a policy clearly helps one industry but hurts another, then other interest groups might simply increase their contributions to change a politician’s vote in their favor – assuming that political spending is truly a paramount factor.

A more straightforward (and logical) proposition is that donors choose to spend their limited resources on candidates that they already support, not on candidates whose minds they need to change first. That is not to say that organizations don’t try to sway politicians through issue advocacy, lobbying, or original research, but one would be hard-pressed to prove that spending money to advance arguments is unjustifiable. This study certainly does not.

III. Conclusion

The authors of this paper – and its enthusiastic backers – declare it to be a groundbreaking work that definitively proves the relationship between political money and policymaking. An examination of what spending the authors consider to be political and how they measure changes in policymaking shows that this contention is far from true. Even more objectionable is how the authors conduct themselves in a fashion that can aptly be described as demagogic, problems with their analysis notwithstanding.

The paper makes numerous insinuations of widespread governmental corruption, while offering only vague anecdotal references as evidence (see footnotes 2, 15, and 16 for examples). Although it excludes such references from its final model, refraining from the use of innuendo as an analytical tool is no virtue, particularly when the writers choose to indulge in it elsewhere. Gratuitous references to “free market propaganda,” “Gilded Age politics,” and “orgies of deregulation” further detract from the study and make it difficult to take seriously as an unbiased analysis of the American political system. Combining such rhetoric with purposeful obfuscation – the vague definitions of what “political” money they’ve actually counted, the burying of illuminating flaws in the footnotes of appendices – makes for a fatal mix.

Interestingly enough, we find some common ground in the authors’ acknowledgement that our expansive campaign finance laws have created a “maze” of bureaucracy and regulation that redirects political spending without actually cutting it off. Reducing the use of money to facilitate political speech is neither possible in a democratic system, nor desirable as a matter of First Amendment principles. If the authors want fewer forms of political spending or fewer politicians bucking party line congressional votes, then the simple solution is to remove limits on campaign contributions. Doing so would strengthen parties, incentivize candidate-centered campaigns over independent expenditures, and simplify the public’s ability to participate in politics.

One should not expect the authors to take this approach, however. Their writing makes it clear that their views are a conclusion in want of evidence, as seen by their frustration with existing research that “facilitates blithe dismissal” of the claims that they feel so strongly to be true. For now, those claims continue to simply be feelings, lacking empirical proof.