To Speed Up Scientific Progress, We Need to Understand Science Policy

Introduction

Science is a key driver of economic growth and social progress. If science can be accelerated — such as by increasing the efficiency with which research dollars translate into discoveries and commercialized inventions — so can growth. Metascience researchers, like us, can generate evidence on the best way to accelerate science. Much is being learnt, but closer partnerships between researchers and policymakers could allow scientists to do much more.

Yet academic findings that affect the practice of science are still the exception rather than the rule. For example, one widely cited paper¹ compared the scientific discoveries of researchers who had received ‘person-specific’ grants with the findings of a matched set of similarly accomplished researchers who were funded by ‘project-specific’ awards. (The former were Howard Hughes Medical Institute investigators; the latter received funding from the US National Institutes of Health.) The person-specific awards produced more novel lines of inquiry and a greater number of high-impact articles. Some science funders — public agencies and private philanthropies — have told us, off the record, that these results led them to increase their use of person-specific approaches to science funding. But in terms of dollars, project-specific approaches to funding remain by far the norm.

How can we do better? By clarifying how structures like funding, career evaluation and peer review ought to change in response to a study. New facts alone cannot directly inform policy. For example, a 2007 paper documenting the rise of team science across a variety of disciplines has, according to Google Scholar, received more than 3,700 citations.² However, in order for this well-cited paper to be actionable for science funders, it needs to be paired with follow-on work clarifying its implications. For instance, tenure — which aims to evaluate the promise of individual researchers — seems well suited to a world in which individual inventors discover new ideas, but is less well matched to a world of team science.

We think that one path forward is to encourage more science-policy research that seeks fundamental understanding of scientific problems while having immediate use for society — a class of research that falls within ‘Pasteur’s quadrant’.³ Just as Louis Pasteur’s work on pasteurization was inspired by observing manufacturers struggling with bacterial contamination of wine and milk, we see value in use-inspired research on science policy. In our view, the current dearth of such work stems from a lack of understanding about which questions policymakers most need answering. Here are three ideas for how to fix that, drawing on our experience working with US federal science agencies.

Spend time working in government to understand how policy works

The current gap in policy-relevant metascience is analogous to the ‘laboratory to market’ translation gap in other fields. This is good news, because other disciplines can offer clues about which solutions might work. For example, Nobel prizewinner Carolyn Bertozzi has spoken publicly about the value of doing chemistry and biology research in close partnership with Stanford University’s medical school in California, which, she argues, shifted her research towards use-inspired questions. At universities such as Stanford University or the Massachusetts Institute of Technology in Cambridge, students regularly see spin-off companies being formed, making clear where the frontier of high-impact translational research lies.

Analogous solutions for science policy would involve researchers building relationships with the institutions their research seeks to inform, such as US federal science agencies. Building such relationships is challenging, because academics and policymakers often don’t speak the same language. For example, Doug Elmendorf, former director of the US Congressional Budget Office, has written about how academics tend to speak in jargon and use technical detail, whereas in his experience, effective communication with Capitol Hill staff requires infographics that clearly illustrate key ideas.

Perhaps the most straightforward way for academic researchers to bridge this translation gap is to spend time in government service positions. Conventionally, in the United States, academics have taken formal positions, including secondments at the Council of Economic Advisers (part of the President’s executive office), as temporary programme officers at the US National Science Foundation (NSF) or as programme managers at the Defense Advanced Research Projects Agency. But in the past decade, many agencies have also designed less-formal positions, such as paid or unpaid part-time or remote assignments through the Intergovernmental Personnel Act or other mechanisms.

It is important to realize that science-supporting federal agencies are not monoliths. The NSF and the NIH, for example, have dozens of decision makers, each with some latitude and discretion to set practices such as how grant applications are solicited, evaluated and distributed. Academics should aim to learn about particular programmes, seeking to understand their goals, constraints and objectives. Following and sending feedback in response to agency requests for information is one path forward.

Government institutions can help by creating more part-time or short-term opportunities to build relationships with academics. One example is a recent NSF award to the National Bureau of Economic Research to create the Place-Based Innovation Policy Study Group — a group of academics, practitioners and NSF staff that aims to deploy “timely insight for the NSF Regional Innovation Engines program”.

The group is led by one of us (H.W.) together with two economists who have deep expertise in entrepreneurship, innovation and regional development — Jorge Guzman at Columbia Business School in New York City and Scott Stern at MIT — who are providing an assessment of the “state of knowledge” of place-based innovation ecosystems and their relationship to geographical and socio-economic inclusion. They are also identifying insights to inform NSF and the broader community on design of NSF Engines, and identifying opportunities to measure and assess place-based innovation ecosystems on a rigorous and ongoing basis.

The structure of this exciting pilot has the potential to encourage the adoption of evidence-based science policies by the NSF, as well as direct academic researchers towards neglected questions. Tellingly, the group was hatched, encouraged and joined by former academic Daniel Goroff, who at the time had just returned to the Alfred P. Sloan Foundation in New York City from a position at NSF’s Social, Behavioral and Economic Sciences directorate. This illustrates how one person’s time spent interacting with practitioners can help to create opportunities for others to connect.

Look for ways to collaborate with think tanks

Academics need not work in isolation, and often benefit greatly from working with institutions that are more experienced in translating academic research into social impact. For example, in 2022, the Federation of American Scientists (FAS; led by D.C.) and the Institute for Progress (IFP; co-led by C.W.) launched science-policy initiatives. These institutions bring skills and experience in connecting social scientists and the federal government and in translating innovative ideas and best practices into policy design. Academics can collaborate directly with think tanks in various ways (for example, H.W. is the IFP’s director of science policy, and P.N. is a senior fellow there).

In 2022, the FAS and the IFP launched the Metascience Working Group, which aims to facilitate metascience proposals, discussions and insights with a focus on projects relevant to the Advanced Research Projects Agency for Health, the NIH, the NSF and the US Patent and Trademark Office. Our motivation in developing this group came from conversations with staff at some US federal science agencies who expressed an interest in engaging with academics and outside practitioners, but who were having trouble juggling outreach and engagement on top of their day jobs.

One goal of this group is to serve as a matchmaker between academics and science funders. For example, the Sabbaticals in Service programme aims to enable matches between academics who are interested in working with federal agencies and agencies that are searching for experts in particular areas. The Metascience Working Group can also make referrals to outside organizations. For example, agencies expressing an interest in randomized evaluations can be referred to the Science for Progress Initiative (co-led by P.N. and H.W. through the Abdul Latif Jameel Poverty Action Lab) for help identifying potential academic collaborators.

Work to change academic norms to value use-inspired research

To be sustainable, use-inspired research must be recognized not just as practically useful but also as a valuable scientific contribution. One example is the work of economists Lauren Lanahan at the University of Oregon in Eugene and Kyle Myers at Harvard Business School in Boston, Massachusetts. When Lanahan and Myers were PhD students in the mid-2010s, their advisers suggested they serve on a National Academies Review of the Small Business Innovation Research (SBIR) programme. Over six months, they provided a rigorous evaluation of the SBIR programme that played an important part in congressional testimonies about the programme. This evaluation also laid the groundwork for several academic papers that Lanahan and Myers continued working on after their service.⁴⁵

Although this example gives reason for optimism, we recognize that some academics might face a trade-off if they choose to pursue use-inspired work that does not result in published academic papers or other conventional metrics of success. In our view, there is hope for change. There is a general perception in academia that all academics value and respect research that has conceptual novelty or significance, but only some appreciate and value its real-world relevance. This is accurate to an extent, although the difference might be less pronounced than some think, suggesting there could be an opportunity to change norms at universities.⁶

Tenure provides some professional insulation for academics to invest the time required to build relationships with science funders, rather than simply publish papers. However, there is some evidence that academics (economists, at least) do not make much use of tenure to change their research trajectories.⁷ By shifting norms about what kinds of research topic are interesting, tenured academics who pivot to work in Pasteur’s quadrant could enable others to follow.

Although use-inspired science-policy work is starting to happen, in our view it remains drastically undersupplied. Our early efforts suggest that both academics and US federal science agencies see value in having more work in the Pasteur’s quadrant of science policy.