Following the rules: Connecting academic research to policy

Key Takeaways

• Academic research can have an important impact on policy through the notice-and-comment rulemaking process, but participation among academics is low. 
• We provide a series of recommendations for academic institutions and federal agencies to support academic engagement in policies impacted by federal regulations.
• These recommendations include developing infrastructure for tracking relevant regulatory actions; providing analytic and drafting/editing support to help produce comments; and innovating incentive structures that can reward academics for public policy engagement.
• Federal agencies can help encourage involvement by academics by standardizing citations of research and public comments in regulations and committing resources to outreach and communication of upcoming regulatory actions. 
• These efforts will help bridge the gap between research and policy and improve the likelihood that policymaking reflects the best available evidence while also helping universities fulfill their mission to increase public impact.

Universities across the country are renewing their missions toward making a public and civic impact.  While academics are producing policy-relevant research, advising government agencies, and contributing to policy debates, their work can even more directly shape policy through the federal regulatory process. Often dominated by lobbyists and those with a financial stake in the regulation’s outcome, there is a thirst — and a need — for objective scholarship in this space.

The Stanford Institute for Economic Policy Research (SIEPR) and the Tobin Center for Economic Policy at Yale University are working together to make it easier for academic research to play a role in the regulatory process.  This policy brief describes how policy is made through regulation, the value that academic research can bring to the process, barriers that prevent participation, and ways that academic institutions and federal agencies can help overcome them.

How is policy made by regulation?

When Congress passes a law, it’s often up to a federal agency to develop the regulations needed to implement it. Regulations fill in details not spelled out in statute.

That regulatory authority is often delegated by Congress and can be explicit or implicit.  For example, the Secure Act 2.0 of 2022 explicitly authorizes the Secretary of Treasury to “prescribe such regulations as may be necessary to clarify” how to treat an employer that fails to meet multiple employer 403(b) plan requirements.  Similarly, the Inflation Reduction Act of 2022 directs the Secretary of the Treasury to issue regulations, as needed, on vehicle classifications, presumably to aid in the determination of the clean vehicle credit. 

In contrast, the 1965 Higher Education Act provided financial aid to students attending postsecondary higher education programs that “lead to gainful employment in a recognized occupation.”  However, “gainful employment” was never defined until 2011 when the Obama administration issued regulations explicitly defining how schools can demonstrate how their programs (at for-profit institutions and certificate programs at public and nonprofit institutions)  lead to “gainful employment,” providing an avenue to address predatory behavior and exorbitant debt experienced by students at for-profit colleges. 

What procedure do federal agencies follow when issuing regulations?

The Administrative Procedure Act (APA) established the process that all federal agencies must follow when issuing regulations, summarized in Figure 1.  This process helps discipline federal agencies in the development of regulations in three notable ways, which also distinguish it from the legislative process:  

• Regulators are legally required to justify policy choices;
• Regulators must seek public comment on a proposed rule, including any information relied upon by federal agencies when establishing the basis for a regulation, and respond to comments in the final rule;
• Both of these aspects of rulemaking can be challenged in court if not appropriately conducted.

Figure 1:  Notice and Comment Rulemaking Process

Sources:  “How to effectively comment on regulations.” Shagufta Ahmed, Shannon Joyce, and Adam Looney. The Brookings Institution, August 2018;  “Learn About the Regulatory Process.” www.regulations.gov/learn.

What are the standards that federal regulations have to meet?

While the APA sets out the requirement for federal agencies to seek and respond to public comment and to justify their regulatory choices, a series of court decisions have further defined how this requirement must be met.

Federal agencies are required to “give interested persons an opportunity to participate in the rulemaking through submission of written data, views, or arguments …” (5 U.S.C. § 553(c)), and courts have ruled that any "technical studies and data" upon which the agency relied must be transparent in the proposed rule (U.S. Chamber of Commerce v. SEC,  2006) or provided through a public comment.  Regulations may be deemed “arbitrary and capricious” and invalidated if they fail to demonstrate “reasoned decision-making” or appropriately account for data and evidence that has been presented to the agency.[1]

Figure 1: Notice and Comment Rulemaking Process

The legal standard of “reasoned decision-making” is often a cost-benefit analysis that evaluates the proposed policy relative to a baseline and is shaped by facts, evidence, and analysis.  In addition, federal agencies must consider and adequately respond to comments submitted by the public.[2] Further, court decisions have defined that only information made transparent by the federal agency when proposing a regulation or submitted through a public comment will be considered part of the official rulemaking record. Outside of these two channels, courts will not consider information, despite its relevance and widespread use, when evaluating a regulatory issue before the court (U.S. Chamber of Commerce of v. SEC, 2006).

The opportunities — and challenges — academic researchers face in the regulatory process

Comments from experts with objective evidence are highly valued and influential in informing the regulation’s outcomes.  However, input in the regulatory process is often dominated by those with financial stakes in the regulation’s outcome — such as representatives from private industry or lobbyists — rather than those with objective evidence and expertise (Golden, 1998; Yackee and Yackee, 2006). 

So why aren’t more academic researchers involved? SIEPR and the Tobin Center conducted a survey of faculty affiliated with each institution and engaged in follow-up discussions with those who agreed to be contacted. Here’s a summary of common themes and findings:[3]

Lack of awareness

Many faculty see value in public policy engagement. When asked about the arena in which they were most interested in having an impact, the vast majority (91 percent) indicated that government and public policy were among their top choices.  Advancing knowledge/academic impact scored similarly (82 percent), with influencing markets and the private sector and improving awareness in the general public as more distant goals (12 and 15 percent, respectively). In addition, almost half were interested in increasing their engagement with public policy.

But a significant share of respondents (45 percent) also indicated little or no familiarity with the federal regulatory process, and many were unaware policymaking occurred outside of the legislative branch. Some were surprised to learn that key policy details of legislation enacted by Congress are often substantially defined by federal agencies through regulations.

Among those who have engaged in the process, many had served in official capacities in the executive branch.  Even those who had engaged several times generally relied upon other parties (e.g., think tanks) to bring relevant regulations to their attention, highlighting the difficulties in keeping apprised of regulatory actions.

Lack of professional incentives

Faculty face several competing demands on their time and generally receive little to no reward for activities beyond the publication of peer-reviewed research papers, teaching and mentoring, and administrative duties within their organizations. 

Producing public comments — and policy engagement more generally — may garner some recognition by some departments. But even in those situations, there was general agreement that any appreciation that was received was not commensurate with the time involved relative to other actions. Those who engaged in public policy tended to exhibit intrinsic motivation in their research having an impact beyond the academic arena.  Faculty were more likely to devote time and effort after having secured tenure in their academic position. 

Uncertain benefits, timing mismatch, and other concerns

Even faculty who really want to engage in public policy were hesitant because of the uncertain benefits. Some wondered whether policymakers would be interested in their input and if their efforts would amount to meaningful change. Several said that examples and evidence regarding the impact of academic engagement would help. 

Others simply did not have the capacity to track when policy actions that could benefit from their research were occurring.  If such an item was brought to their attention, it was unlikely that a researcher would be able to provide adequate attention in the short window available.

Faculty often said policy engagement felt intimidating and hard to navigate. Some hurdles included a lack of understanding of (1) effective entry points, including stages in the rulemaking process where engagement would be worthwhile; (2) key players in the policymaking arena and how relationships can be developed with them; (3) how to effectively engage in policy engagement venues; and (4) which policy issues are active and open for engagement.  

Finally, some faculty said communicating to non-academic audiences presented challenges. Some were concerned that their input could be misinterpreted or taken out of context.  Others felt that guidance — on how to structure and format a well-written comment for a policy audience and how to translate research in a policy setting — would be valuable.

How can institutions encourage more involvement by researchers in the regulatory and policymaking process? 

Universities and federal agencies can make changes to help address these barriers and inject more evidence into the regulatory process. 

Academic institutions

American universities are increasing their focus on making a public and social impact in the world.  For example, Stanford’s vision states, “the scale and urgency of challenges we face in society today impel us to ensure ideas born at Stanford benefit the widest population.” Similarly, an aim of Yale’s strategic initiative is “to share light and truth — to increase knowledge and understanding of pressing domestic and international challenges.”

Universities understand that improving society requires more than publishing and teaching. But at most institutions, academics aren’t evaluated or rewarded for their public service.  While there is no “one-size-fits-all” solution, we suggest structures that reward inputs in the policy process over outcomes.  This structure acknowledges that research and evidence are important parts of the policy process, but not the only ones, and that measurement of the impact of one action relative to a counterfactual is difficult, if not impossible.

With the necessary support, federal rulemaking offers tremendous opportunities for academic impact. Many faculty would benefit from institutional help in the process of curating and tracking policy and regulatory developments and matching them with faculty research; directed support of research assistants and specialized staff in drafting and generating public comment pieces that translate research into policy positions; and editing resources to ensure that written materials are appropriate for the target audience and packaged in the most effective way.

We acknowledge that these types of structures require a prioritization of resources that may involve difficult trade-offs.  However, the return on these investments can have a direct and profound impact on policy.

Federal government agencies

Efforts among academic organizations to reward policy engagement require the ability to measure inputs in the policy process.  Academic citations provide a measure of the influence of research in academic literature, but efforts to measure citations in policy settings are nascent and not comprehensive. 

The regulatory process provides a promising avenue to acknowledge the input of the research community, particularly given the potential to measure citations of research studies and public comments in the rulemaking record.

Establishing and enforcing a standard citation approach across federal agencies could foster researcher participation in rulemaking, while also improving the evidence base for regulations and improving compliance with the APA’s transparency requirements.

Absent standard citation guidelines, the manner in which research and comments are cited in regulations varies significantly across federal agencies.[4] This makes it difficult to see how research informs regulatory policy, for academic institutions to reward and incentivize engagement in the process, and for tools that aggregate citations in government documents and other policy settings to emerge.

Federal agencies can play an important role in educating the academic community about the regulatory and policymaking process and how academic research can improve the evidence-base for policy decisions. Devoting resources to outreach at conferences and academic institutions about the value of evidence in policy and ensuring regulatory agendas and items are clear and easy to find can help ease the barriers to participation described above.

These actions are also responsive to the Biden administration’s Executive Order on Modernizing Regulatory Review, which directs federal agencies to proactively engage “those with expertise in relevant disciplines'' in the regulatory process including the development of regulatory agendas and plans.

I’m a researcher who wants policymakers to use my research.  What can I do?

 Participation in the notice and comment rulemaking process is a great way for research to enter the policymaking process because of its legally enforced standards for reasoned decision-making, public participation, and transparency.  But navigating the process can be complex and ensuring comments are most effective takes practice.  Below, we offer some suggestions to address these concerns.

How do I find regulations in my area of expertise?

All federal regulations must be posted on the Federal Register.  Create an account on the Federal Register and then sign up for subscriptions to be notified of regulations open for feedback.  Each agency is also required to publish its regulatory agenda, which provides a short- and long-term forecast of upcoming regulations and is updated every six months.  Regulatory agendas are published on reginfo.gov. 

All regulations will have a preamble that provides a plain language narrative that precedes the legal text of the regulation.  Many regulations will have a dedicated regulatory impact analysis section that researchers can look to for the analysis of the estimated costs and benefits of the regulatory action. 

What kinds of comments can make an impact?

Writing a thorough comment can be daunting and time consuming, but the perfect need not be the enemy of the good.  Comments can be relatively short and do not need to address all aspects of the regulation. Feel free to stick to your area of expertise. 

Proposed rules may pose questions to the public and can form the basis for replies and comments.  Responding to these queries directly is likely to be extremely valuable in the development of the final regulation, as targeted questions often indicate areas where regulators are still not clear on which policy direction to take.

Academic researchers are well equipped to provide feedback on literature cited or not cited by regulators and the methods and data used to inform regulatory choices. The feedback can be specific to the approaches used or the data the agency relied upon. Alternative approaches or data sources that can be used that are not subject to the same criticism can help agencies revise the regulatory approach in the final regulation.

Additionally, submitting existing research relevant to the policy under consideration through the public comment process makes it officially part of the rulemaking record, therefore ensuring that it has an opportunity to be considered by regulators when finalizing their regulatory approach and strengthens the evidence-base for the regulation. Comments that serve to correct or elaborate on the characterization of research already cited by policymakers are also highly valued.  Any research can be valuable — peer-reviewed articles need not be published only in highly ranked journals, and working papers can also be cited in regulations.

It is important to note that comments can support the regulator’s approach or be critical of it.  Supportive comments can serve to validate the research cited by the agency on the record and can affect the course of the regulation if the action is subject to legal challenge.  Showing consensus among a number of experts can be particularly effective, making co-authored public comments valuable.

The credibility and objectivity of the commenter is weighed heavily in the regulatory process.  So  it is important to provide your credentials and expertise when commenting and to disclose any potential conflicts of interest.  Guides provided by academic journals and research organizations can be helpful in determining which activities should be disclosed.

Finally, the audience for public comments often includes specialized public servants who are professionally trained in economics.  Therefore, the hurdles to communicating the way that a research study applies to the rule are generally lower than writing for a more general audience.

Conclusion

The federal rulemaking process has the potential to be a powerful avenue for academic engagement. It has existing structures in place that not only mandate public feedback but also compel policymakers to pursue policy decisions that are well justified.  Academic experts have a significant comparative advantage in contributing to the rulemaking policy arena.

Given the potential for impact, universities and federal agencies should consider institutional structures and other support that could lower the barriers for academic engagement in the regulatory process.

Footnote

[1] Courts ruled that an “agency rule would be arbitrary and capricious if the agency … offered an explanation for its decision that runs counter to the evidence before the agency …” (Motor Vehicles Mfrs Ass’n v. State Farm, 1983).

[2] See, for example, Business Roundtable and U.S. Chamber of Commerce v. SEC, 2011.

[3] The survey had a low response rate (38/208) and thus may not be representative of all affiliated faculty.

[4] See, for example, the Centers for Medicare and Medicaid Services (CMS) proposed rule, “Patient Protection and Affordable Care Act; HHS Notice of Benefit Payment Parameters for 2023,” which included a two-stage weighted model specification to recalibrate the adult and child risk adjustment models to improve the underprediction of plan liability for the lowest-risk enrollees. After reviewing the public comments on this issue, CMS decided not to finalize the adoption of the two-stage weighted model specification. In a public comment, researchers Matthew Fiedler (Brookings Institution) and Timothy Layton (Harvard University) provided analysis and research, which explained the anti-competitive effects of the two-stage approach: “In ongoing work, one of us (Layton) has found that, in markets with many competing issuers, the resulting “race to the bottom” in premiums can cause all insurers to incur losses. As a result, insurers will be forced to exit until just one or two insurers remain, leaving consumers with high premiums and little plan variety.” While the authors of the public comment cited this and other research to justify their recommendations, and CMS acknowledged the anti-competitive concerns raised by commenters, CMS did not cite research provided by public commenters when it ultimately made this change in the final rule.

References

Golden, Marissa Martino. 1998. “Interest Groups in the Rule-Making Process: Who Participates? Whose Voices Get Heard?” Journal of Public Administration Research and Theory 8(2):245–270.

Yackee, Jason Webb, and Susan Webb Yackee. 2006. “A Bias Towards Business? Assessing Interest Group Influence on the U.S. Bureaucracy.” The Journal of Politics 68:128–139.