2.6: Reporting, Verifying Scientific Studies

Research results are published in scientific journals, making them available to scientists and others throughout the world. Scientists scrutinize studies and refer to them in follow-up studies that verify or build on the results. When a finding is consistently verified by other scientists, it’s accepted into the body of scientific knowledge (otherwise, it’s excluded). Lavoisier’s studies in the 1780s supported his hypothesis that animal life depended on oxygen. Others verified his findings that now are an accepted part of scientific knowledge.

The results of scientific research are usually reported in a standard format:

• Introduction: Includes the hypothesis and background information to orient the reader as to why the study was done.
• Methods: Describes in precise detail the materials (e.g., diet, animals) and methods used. Details are crucial; scientists in other labs must be able to duplicate and validate the results of the study
• Results: Typically presented in tables and graphs that include statistical analyses.
• Discussion and Conclusion: Results interpreted and compared to other relevant studies.

Thousands of scientific journals are published throughout the world; they vary in subject matter, quality, and distribution. Many are specialty journals (vs. the more general, interdisciplinary ones). Some titles: Sexually Transmitted Diseases, Lipid Research, Cell, Anesthesiology, Cancer, Aerospace Medicine and Biology, Clinical Chemistry, Blood, Gut (not Blood and Guts!), American Journal of Clinical Nutrition, Atherosclerosis, AIDS Research and Human Retroviruses, and Pediatrics.

Among the most prestigious and widely read are two medical journals—The New England Journal of Medicine (nejm.org) and The Lancet (thelancet.com)—and two general science journals—Science (sciencemag.org) and Nature (nature.com). These four are all:

• In English, the most commonly used language in science. Journals in other languages often include English abstracts of their articles.
• Widely read throughout the world. Subscribers include many individuals, libraries, companies, college departments, etc.
• Published weekly (most are published monthly). Traditionally, credit for a discovery goes to whoever publishes it first. As in the awarding of patents, this can be a very close and rewarding race. (Who remembers who applied for the patent for the telephone a few hours after Alexander Graham Bell? Not many have heard of Elisha Gray.)
• Peer‑reviewed. Scientists submit their studies, which are reviewed critically by their peers—other scientists in the same field. The articles are then rejected or accepted for publication. Even the accepted ones may be sent back for revision, e.g., to include discussion of a relevant study published earlier. The best journals are peer-reviewed. At the other extreme are journals that for a fee uncritically publish virtually any study; these don’t generally attract high-quality research articles.
• Very selective in accepting only those articles considered the most timely and important. Getting an article into these journals is very competitive, so it’s prestigious to get published in them—the journals’ success breeds success. Most of the breakthroughs in AIDS research were published in these four journals.

In 1953, Watson and Crick published their discovery of the structure of DNA in Nature (they won the Nobel Prize in 1962).

In 1996, Richard Zare and his colleagues published in Science their evidence of microbial life 3.6 billion years ago on Mars.

In 2001, the two teams competing to complete the first draft of the Human Genome Project (see Chap. 10) published their results simultaneously: Francis Collins’s group in Nature (2/15/01), and Craig Venter’s group in Science (2/16/01).

In 2012, Jennifer Doudna and Emmanuelle Charpentier published in Science the key study demonstrating the revolutionary gene-editing technique CRISPR-Cas9 (see Chap. 10), and shared a Nobel Prize in 2020.

In 2020, Most of the breakthrough studies on the virus SARS-CoV-2, and COVID-19, the disease it causes, were published in these four journals.

Besides publishing in journals, scientists present their results at scientific meetings. For example, 10,000+ scientists met in San Diego in 2016 for the 100th annual Experimental Biology meeting, where thousands of studies were presented. After the scientists present their studies, fellow scientists ask questions and make comments (peer review). The meeting provides an opportunity for dialogue among scientists. They even have dinner meetings based on their research interests—a Liver Dinner is not a dinner of liver!

Claim of Cold Fusion

The traditional way in which scientists report their findings is occasionally breached. The widely publicized claim in 1989 by chemists Stanley Pons and Martin Fleishmann that they produced energy by cold atomic fusion provides an insight into the practical importance of the scientific method for both scientists and nonscientists.

The claim was astonishing because of what was known about atomic fusion. We’ve all heard that what’s too good to be true is usually untrue. As will be discussed in the next chapter, physicists have been working for decades on hot fusion— using extremely high pressure and temperature to fuse two hydrogen atoms to make one helium atom to get a virtually unlimited supply of energy.

The two chemists first reported their results at a press conference rather than at a scientific meeting or in a science journal. Because they didn’t reveal the details of their experiments, other scientists couldn’t evaluate their validity. Scientists traditionally withhold judgment until they can evaluate the data, so their public responses were subdued (“I’d be very surprised if it were true”). In sharp contrast was the exuberant response of many nonscientists who discussed the revolutionary consequences of cold fusion (e.g., solving the energy crisis and global warming) and put the two chemists on the cover of Time.

Other aspects of this episode shed some light on the Scientific Method and interactions among scientists:

• The day after their press conference, the chemists submitted a report of their study for publication in Nature. It was peer-reviewed, returned for substantial revision (the report was lacking in essential details), and wasn’t published in Nature. But another report they had submitted earlier to the Journal of Electroanalytical Chemistry was published shortly thereafter. Since then, many scientists have tried to duplicate their results, but generally without success.
• If Pons and Fleishmann had found an inexpensive way to produce energy by cold fusion, the commercial interest and the financial gain would have been substantial. This may help explain why they were reluctant to reveal the details of their experiments. Traditionally, scientists freely share their materials and data with other scientists. Ties between scientific and commercial interests can put a damper on this. Because of commercial interests and the increased scarcity of government funds, more university scientists (especially in biotechnology) are getting funds from industry.
• Nuclear fusion is more in the realm of physics than chemistry. Pons and Fleishmann are chemists. Because of the burgeoning knowledge in science, scientists have had to markedly narrow their research—scientists need to “know more and more about less and less” to make advances in their fields. It thus becomes less and less likely that chemists will make major discoveries in physics and vice versa—Pons’ and Fleishmann’s claims were much better received by their fellow chemists than by nuclear physicists.

It’s increasingly hard for scientists to understand the intricacies of research in fields other than their own. There’s the story of a chemist and a physicist who attended a lecture by a distinguished mathematician. Afterwards, the chemist said, “I lost the thread of the lecture about two minutes from the end.” The physicist replied, “That’s amazing; I didn’t understand any of the lecture at all.” The chemist replied, “Well, I didn’t say which end I was referring to!”