A Study Says… But what does that really mean?

Scientific papers are supposed to be written by experienced researchers and submitted to journals that should guarantee a given level of quality. To ensure proper quality, journals recruit specialists in the field to assess paper quality, and based on the comments, the work is approved or rejected for publication. This process is called peer review.

But over the years, it has become clear that simply citing a paper is no longer enough to win an argument. If anything, it is more important that the public understands how to place a scientific paper in context and what its real relevance might be.

Why We Should Be More Careful with Scientific References

Around 3.4 million papers are published annually. The Nature Index 2024 shows the US and China dominate publications, while Portugal ranks 29th in absolute numbers. However, countries like Pakistan, Vietnam, and India lead in the number of papers per capita. Scientific publication is rising globally. This could mean more countries are investing in research, more scientists are working on important questions, or there is a greater curiosity. However, it poses challenges: how can we evaluate millions of papers and trust their results? Retracted papers are a worrying trend. Publishers remove publications with serious issues, such as misconduct, methodological flaws, or ethical concerns.

The situation becomes even more complicated when papers are cited in public discussions. It is common to see opposing claims both supported by “studies.” A well-known example is a 1998 paper suggesting a link between the MMR vaccine and autism spectrum disorder. The study received widespread media attention and contributed to a decline in vaccination rates. Later investigations revealed serious methodological errors, conflicts of interest, and ethical violations, and the paper was eventually retracted. Despite this, some public figures still cite it, repeating claims long discredited by the scientific community.

Importantly, when a paper is later refuted, it does not necessarily mean scientists were dishonest. Often, it reflects how science progresses: through multiple studies testing the same question from different angles.

Why So Many Papers Are Published

Modern science’s organisation many times pressures researchers to fabricate or manipulate results to secure funding and maintain academic positions. This leads to Paper Mill schemes that produce articles with fabricated data for a fee or exchange authorship accreditation to boost publication numbers. Predatory journals offer publication with minimal or no real peer review to maximise profits. Even without misconduct, incentives subtly influence research. Replication studies, which verify the reliability of previous results, are rarely feasible due to the difficulty in securing funding. Finally, commonly, negative results are unpublished because journals lack interest in accepting such results, as they prioritise transforming evidence.

Does This Mean Science Cannot Be Trusted?

Of course not. Due to its self-correcting nature, science remains the most reliable method we have for understanding the world. A single paper does not define its strength, but it contributes to the collective process of testing, replication, and validation. The scientific process is thus able to correct itself over time by discovering possible flaws, re-doing experiments to validate or refute a previous hypothesis. In that sense, what appears true and compelling today might be discredited tomorrow. For instance, between the 17th and early 20th centuries, tobacco was advised to treat asthma and other respiratory diseases. At the time, based on the best available evidence, doctors’ knowledge pointed in that direction. However, we now know better that it not only does not treat but also worsens those conditions. This constant correction, although it might seem contradictory, is how knowledge progresses.

Types of Scientific Papers

When people cite “a study,” they often imagine an original research article. However, this is not the only type of scientific writing. Scientific literature comprises various papers with distinct purposes. The most common are original research articles presenting new experiments or data. Exploratory studies investigate new or overlooked research directions, often concluding with a structured hypothesis and a call for further research. Confirmatory research follows a stricter structure, designed to test a specific hypothesis.

Although most people think of papers as confirmatory studies, I would say the majority (depending on the field) are exploratory. In either case, these studies address specific questions, so any single study alone rarely provides a definitive answer.

Review articles summarise results from many studies on a topic, providing a clearer picture of the scientific community’s understanding. Most reviews are narrative and lack a defined search strategy, which can reflect the authors’ intent. Structured approaches include systematic reviews and meta-analyses. Systematic reviews follow a defined methodology for searching the literature, including keywords, databases, and criteria. Meta-analyses collect quantitative data from studies and analyse it statistically to determine the direction of the combined evidence.

Finally, researchers could discuss interpretations rather than presenting new results in commentaries or opinion pieces. Recognising the type of paper helps judge the weight of its conclusions.

Can We Tell if a Study Is Trustworthy?

It would be convenient if every scientific paper came with a reliability score. Some tools already try to do something similar by analysing citations, checking statistical consistency, or evaluating whether methods are clearly reported. However, judging research reliability based solely on the paper is more complicated. The strength of a study depends on factors such as experimental design, sample size, and how the results fit with previous work.

Even so, a few simple questions can help place a study in context. One is whether the claim relies on a single paper or on multiple studies that reach similar conclusions. Scientific knowledge tends to become stronger when different research groups obtain comparable results over time. You can look for this in the citations in the introduction and discussion of the paper. It can also help to look at how the study was conducted. Very small samples, lack of control groups, or vague descriptions of the methods can suggest that the results should be interpreted with caution. To gain insight into these topics, look at the methods and the graph subtitles. Finally, it is worth asking whether the result fits within a larger body of research. A study that aligns with consistent findings across many papers generally carries more weight than a surprising result that has not yet been confirmed. To check for this, you might need to have a look at scientific databases, such as PubMed, Web of Science, or other more common databases in the field of study. For this reason, scientists rarely rely on a single paper, but rather, on the accumulation of evidence over time.

In the end, a scientific paper is not a final answer. It is a contribution to an ongoing conversation about how the world works. So, the next time someone says that “science says” or that “a study proves” something, it may be worth pausing for a moment. One paper alone rarely settles a debate. Learning to read scientific articles this way may be the most useful step toward navigating the overwhelming landscape of modern publications.