Financial Crises Teach Us That AI Self-Regulation Must Not Replace a Broader Regulatory Framework

Who gets to decide when and how AI models are deployed? The sudden deluge of alarming news stories about AI agents—from hacking computer systems without instruction to being used to augment traditional weapons systems, make military decisions, or even develop bioweapons—has made this an urgent political and policy question. The critical challenge facing policymakers is how to regulate AI in a way that meets the moment without locking in a system that benefits a few incumbent players and their investors at the public’s expense.

AI clearly needs more regulation. One option proposed by some policy experts and industry participants is the creation of some sort of a self-regulatory organization, or “SRO.” When people talk about so-called self-regulation, it can mean different things. The Treasury Department is reportedly interested in an SRO for AI modeled after the privately run Financial Industry Regulatory Authority (FINRA), which oversees the securities industry. Some states and industry participants have proposed using private third-party validators to check the integrity of AI companies’ work. 

We know that pure industry self-regulation is a failed governance model—look no further than the 2008 financial crisis as proof. But are there versions of an SRO with real oversight and accountability that could work? And if so, what are the features that any responsible policymaker should demand? 

As with most things, the devil lies in the details. Fortunately, looking at financial regulation, where there is a history of SROs and other forms of private-sector gatekeepers, can inform the AI debate. Code and money both move and evolve quickly, and AI, like finance, has the potential to become a systemically important part of the economy. The track record of SROs in the financial sector can help us evaluate whether this model is fit for purpose in the context of AI.

What the financial system shows us is that an SRO for AI could have some value, but only if it is appropriately structured and plays a narrow role within a broader regulatory framework with real democratic accountability. In other words, self-regulation can be a small part of a broader regulatory framework—not a substitute for it.

The critical challenge facing policymakers is how to regulate AI in a way that meets the moment without locking in a system that benefits a few incumbent players and their investors at the public’s expense.

Defining Self-Regulation and Its Benefits

Many SROs are themselves typically overseen by a government agency. In the case of FINRA, that’s the Securities and Exchange Commission (SEC), which reviews FINRA’s rules and examines its operations. In this sense, some self-regulatory bodies are more akin to quasi-public, quasi-private entities to which the government has delegated a measure of its oversight powers.

Why would the government want to delegate its authority? The first reason is that SROs or private industry bodies have funding streams that lie outside the congressional appropriations process. This means the entity should have more ample resources and its planning should be less volatile—a clear opportunity when dealing with an AI industry that seemingly mints multimillionaires by the day. An AI SRO may also be able to pay higher salaries to attract more talent (FINRA’s CEO, for example, is notoriously well compensated). 

The ability to attract talent and expertise, coupled with the greater closeness to regulated industry actors, yields a second potential benefit: the ability to access and assess relevant market information in a timely and efficient way. 

Finally, because SROs are not public actors in a strict legal sense, they are able to act more quickly—and more harshly—when doling out punishment. This includes levying penalties like barring bad actors from the industry when they violate industry standards. On the other hand, some of those powers are not necessarily unique to SROs: Some public agencies, such as banking regulators, have similarly broad powers to examine a bank’s operations and books at almost any time, disapprove of novel products banks want to offer, and remove officers and directors and bar them from the banking industry.

One purported legal benefit of an SRO is that it allows for collective, industry-driven measures to address issues of safety without running afoul of antitrust laws against competitors coordinating with one another. But ordinary government regulation can also solve coordination issues without antitrust concerns—and with less of the potential risks.

Potential Problems with the SRO Model

There are at least four potential risks with this approach, though, that echo the experience of financial regulation. 

First, while SROs can test things like software bugs or protocol limitations, they aren’t well situated to deal with larger and more fundamental questions. A central failure of the financial supervisory framework prior to the financial crisis was regulators’ inability to see the forest of systemic risk for the trees of individual banks’ financial health. Financial supervisors have still not fully developed a framework for addressing structural issues. It will be even harder for regulatory organizations with close ties to the AI industry to address profound questions about how that industry should operate in the public interest and avoid truly existential risk. This context should inform whether or not we feel comfortable entrusting such questions about AI to quasi-private bodies.

Second, as a matter of political economy, the outsized influence industries enjoy over their government regulators, known as “capture,” is a long-standing concern. The SRO model’s close connections with the industry it regulates make SROs even more vulnerable than public agencies to industry influence and, ultimately, capture. 

The well-documented example of credit ratings agencies offers a cautionary tale about capture. The credit ratings agency model made ratings essential to Wall Street’s process of packaging subprime mortgages into securities that it then sold to investors. But because the ratings agencies were compensated by securities issuers—i.e., Wall Street banks—they had an incentive to inflate the ratings to attract repeat business. The ratings agencies were also viewed as less sophisticated and lacking the human talent of the Wall Street firms they were supposed to keep in check. As a result, financial markets were flooded with structured mortgage–related financial products whose ratings vastly understated their risks. A financial crisis ensued, and the rest is history.

Third, because the SRO model relies on resources and engagement from industry, it can only work if the industry being regulated is interested in subjecting its practices to high standards. Does today’s tech industry meet that threshold? OpenAI CEO Sam Altman is certainly making the case that the “world should trust that we are going to do the right thing because it’s the right thing and because we feel the magnitude of this.” But even if one was inclined to trust AI executives—which a majority of the public is not—there is a very real risk that the AI industry views its proposals as an opportunity for window dressing rather than real reform.

Fourth, at a more basic level, do we even have the capacity in today’s world to establish a non-captured, self-regulatory body? The fight over the creation of the SEC provides one illustration of how difficult it can be to stand up any type of regulatory structure. The SEC was borne out of the failures of the securities industry’s largely unregulated model prior to the 1929 stock market crash. The securities industry nonetheless lobbied fiercely against the proposed SEC. The Franklin D. Roosevelt administration overcame the opposition, but it wasn’t easy. 

Today, the concentration of wealth in the economy—and the tech sector in particular—is even worse than it was in the 1930s. This makes standing up a new SRO with meaningful ability to check the industry an extremely heavy lift. One way that financial regulation has overcome this risk of capture is by preserving states’ role in regulation, alongside or in lieu of federal action. States can act when the federal government is inert, and it’s harder (though not impossible) to capture 50 different state regulators.

Potential Legal Challenges to the SRO Model

In addition to the policy and political reasons to tread carefully with an SRO model, it also faces potential legal obstacles. In its early years, the SEC barely made it through a legal challenge during another era when the Supreme Court majority was anti-regulation. The same types of claims might be brought against a new AI SRO, and the Roberts Court’s recent moves may increase their likelihood of success.

Recent Supreme Court administrative law decisions and doctrines have limited agencies’ regulatory powers, reduced the independence of the leadership of financial regulatory agencies, and empowered the financial industry to litigate against agencies’ actions. This line of jurisprudence opens an SRO model delegating public functions to private actors to potential legal challenge. That’s because SROs do not just allocate power from the executive and the legislature to a purportedly independent government agency—they arrogate that power to a quasi-private actor.

Regulation is fundamentally about power—who has it, how they use it, and who benefits from it. The public must be the ultimate decision-maker about the present and the future path of this emerging technology.

Model Principles for AI Oversight

So where does this leave us? First, the efficacy of an AI regulator depends on the scope of its responsibilities, the powers it possesses to regulate AI companies, and regulators’ incentives to either exercise those powers or give the industry a pass. The most determinative factor is that the AI industry cannot be in charge, either directly by picking its own regulator or indirectly by serving as its regulator’s primary source of compensation.

Second, democratic oversight and accountability are essential. Whether it’s to oversee an SRO or the AI industry directly, government agencies must build their own independent expertise, capacity, and resources. Rulemaking, standard-setting, and governance processes must also be structured so that they incorporate input from the public, especially skeptical and contrary opinions. Good policy is not made in an echo chamber. 

Finally, the design of SROs must be intentional and thoughtful in whether and how SROs deal with large and consequential issues. Certain structural questions must be resolved by entities with political accountability, not private or even quasi-private actors. 

Some sort of self-regulatory organization might be useful to address the emerging risks of AI. But its role should be narrow, as one part of a broader effort driven by democratic institutions.

Regulation is fundamentally about power—who has it, how they use it, and who benefits from it. The public must be the ultimate decision-maker about the present and the future path of this emerging technology.

Related Resources