Scientific research is conservative but could be accelerated, analysis finds

Computational study of millions of articles shows lack of risk-taking slows discovery, but alternative approaches could unleash it

November 9, 2015

Institutional and cultural pressures lead scientists to avoid risk-taking and choose inefficient research strategies, two new University of Chicago papers conclude. Despite increased opportunities for groundbreaking experiments, most scientists choose conservative research strategies to reduce personal risk, which makes collective discovery slower and more expensive.

However, these computational studies also uncovered more efficient approaches for maximizing discovery and identified the approaches used more often by scientists who have won Nobel Prizes and other prestigious awards.

Together, the studies in the Proceedings of the National Academy of Sciences (PNAS) and American Sociological Review (ASR) quantify the advantages and disadvantages of modern science -- and propose steps for a more productive future.

"The idea here was really to figure out how much scientific activity is innovative and changes the contours of the field, and how much is traditional and reinforces established understandings," said James Evans, associate professor of sociology at UChicago and director of Knowledge Lab at the Computation Institute (CI), a joint initiative of UChicago and Argonne National Laboratory. "The institutions of science reward scientists for incrementally extending existing knowledge, even in the face of exploding opportunities. We find that this leads to inefficient exploration of the space of discoveries, especially as fields mature."

Both papers drew upon new scientific knowledge networks extracted from millions of biomedical journal articles and patents. Co-authors Evans, Andrey Rzhetsky, and Jacob Foster of the University of California-Los Angeles (with the addition of CI Director Ian Foster in the PNAS paper) categorized articles and patents based on the molecules used in the research, with a network link created between each pair of molecules that appear together in the same document.

The resulting networks, made up of thousands of molecular "nodes" and millions of links, allowed the researchers to determine the strategy used by each scientific article. Did it report a novel relationship between two molecules, creating a new link in the network? Or did it replicate a previously known link? The spontaneous organization of the network into "knowledge clusters" corresponding to scientific fields allowed researchers to detect whether a new link connected distant, previously unrelated molecules or consolidated neighboring entities within a single cluster. Researchers also used networks to measure the pace at which new knowledge was revealed.

"By looking at how combinations of chemical names occur and evolve in millions of publications over time, we can model scientific knowledge as a network of connections between important molecules," said Rzhetsky, professor of medicine and human genetics at UChicago, CI Senior Fellow and director of the Conte Center for Computational Neuropsychiatric Genomics. "This allows us to look at how researchers currently work to uncover this network, and what optimal strategies might be."

The first paper, published in PNAS, used a knowledge network to determine the efficiency of scientific research; for example, by measuring by how many experiments were necessary to uncover critical new knowledge. Historically, the analysis found, research within a field grows more conservative over time, with scientists focusing more heavily on well-studied, central molecules.

Conversely, more efficient strategies -- determined by testing thousands of different strategies on the University of Chicago's Beagle supercomputer -- take the opposite approach, with experiments growing riskier and seeking more distant connections over time. If high-risk research is better incentivized, increased publication of failed experiments will also accelerate the pace of discovery, the researchers found.

"Scientists can often get trapped by concentrating on a small part of the network and spending large amounts of resources trying to solve the same problem," Rzhetsky said. "This works for new fields, where many experiments have a high chance of successfully revealing a new connection. But much more effort, time and resources must be spent to make new discoveries in well-established fields. To maximize the pace of successful scientific advances, the best approach is to be adventurous and explore as broadly as possible."

In the ASR paper, the researchers tested the "essential tension" of science: the balance between incremental, conservative research and innovative, novel strategies. The authors found that scientists were six times more likely to perform "repeat" research than studies that created new links between chemicals -- a proportion that remained stable over the 25 years studied despite an exploding number of new research opportunities.

If published, innovative papers that establish new links were more likely to be cited, with a broader variance in citations and a higher average citation count than more traditional findings. Furthermore, papers by authors who won the Nobel prize or other prestigious science awards introduced new molecules and relationships much more frequently. But the authors argue that these additional rewards still do not balance the greater risks of innovative research.

The sustained preference for conservative research, despite greatly expanded access to new molecules, methods, and collaborations and the chance for greater rewards, suggests that institutional structures incentivize lower-risk research. For example, a young researcher pressured to publish frequently will favor incremental experiments more likely to be accepted by journals.

"If we want to push that risk, then we'll have to change the recipe," Evans said. "We'll have to reward at the group level, like Bell Labs did in its heyday, or fund individual investigators independent of the project, so they can intelligently allocate risk across their personal research portfolios."

"Tradition and Innovation in Scientists' Research Strategies" by Jacob Foster, Andrey Rzhetsky, and James Evans was published online September 1, 2015, by American Sociological Review. "Choosing Experiments to Accelerate Discovery" by Rzhetsky, Jacob Foster, Ian Foster, and Evans will be published online November 9, 2015 by Proceedings of the National Academy of Sciences. Both papers were supported by NSF grant SBE 0915730 and the John Templeton Foundation; the PNAS paper was also supported by NIH grants 1P50MH094267 and U01HL108634-01, DARPA Big Mechanism contract W911NF1410333, and AFOSR grant FA9550-15-1-0162,