Although conceptually simple, the term “scientific consensus” is often misused and misunderstood. It can get confused with appeals to popular opinion or erroneously conflated with “consensus” in the colloquial sense of the word. These misunderstandings can lead to things like opinion polls, often predominated by unqualified individuals, being misconstrued as evidence that no scientific consensus exists on some topic for which it clearly does, or that it leans towards a different conclusion than it actually does. In some cases, the very concept itself invokes resentment or even retaliatory commentary from people whose views are threatened by its implications. The purpose of this article is to clarify the concept that the term scientific consensus is meant to refer to and address some of the arguments commonly leveled against it.
Defining Scientific Consensus
Just as the term “theory” has a different meaning in science than its colloquial usage, the term scientific consensus means something different than “consensus” in the usual colloquial sense. The latter typically refers to a popular opinion, and needn’t necessarily be based on knowledge or evidence. On the other hand, a scientific consensus is, by definition, an evidence-based consensus. A convergence of the weight of existing evidence is a prerequisite which distinguishes a knowledge-based scientific consensus from mere agreement. This is critical, because the scientific enterprise is essentially a meritocracy. As a result, it doesn’t matter if a few contrarians on the fringe disagree with the conclusions unless they can marshal up evidential justification of comparable weight or explain the existing data better. The weight of the evidence is paramount.
In a nutshell, a consensus in science refers to a convergence of many independent lines of high quality evidence all leading the majority of active scientists in a given field to arrive at the same conclusion and/or complimentary conclusions. It’s not something any scientist necessarily sets out to become a part of as a goal, but is rather something they discover they’re a member of because that’s where their research results led them. The process by which scientific consensus emerges over time can be complicated and tends to vary from case to case, but it is likely to exist whenever scientific knowledge is this best explanation for a given consensus. Furthermore, scientific knowledge is the best explanation for a consensus when the following definitional criteria are satisfied:
Via Denial 101
Consilience of Evidence: The consensus should be based on varied lines of evidence which independently converge on the same conclusion or set of conclusions [1]. The scientists and their results needn’t necessarily agree on every single minute detail, and the data convergence will typically fall within a set of error bars, but will point to the same general conclusion even if debates still exist on the minutia. This may involve contributions from multiple scientific sub-specialties, each providing different pieces comprising a broader understanding or set of conclusions.
For instance, the scientific consensus in climate science incorporates evidence and expert knowledge from meteorology, geology, geophysics, geochemistry, atmospheric physics, atmospheric chemistry, community and global ecology, astronomy, planetary science, and even stellar astrophysics. Scientists from different specialties study different aspects of the issue and arrive at results comprising a piece of a puzzle whose results are all consistent with the conclusion that the recent warming trend has been largely the result of human activities [38],[39],[40],[41],[42].
Similarly, the scientific consensus on the individual and social benefits of vaccination combines knowledge and evidence from fields such as microbiology, immunology, virology, epidemiology, systems biology, molecular biology, biochemistry and more. Knowledge comes together from these disparate disciplines to create vaccines that significantly reduce the likelihood of their recipients contracting the diseases against which they are designed to protect, whose risks are greater than the minuscule risks of adverse reactions to the vaccines [36],[37],[43].
Social Calibration: The experts involved are mutually committed to employing the same high standards of evidence and formalisms, and have good justifications for those standards [1]. Nobody disputes that carefully collected and reproducible evidence is key in science; the problem is that evidence doesn’t talk. It has to be interpreted by human scientists. The Social Calibration criterion has to do with what the scientific community as a whole accepts as evidence, how they decide what is relevant and significant, and how individual scientists persuade their peers that they are correct.
One of the reasons that certain fringe disciplines are viewed as pseudosciences by mainstream scientists is because they operate under lower and/or inconsistent standards. For example, one of the most important methods of ascertaining the safety and efficacy of a given medical intervention in conventional science-based medicine is the performance of a large randomized double blinded placebo-controlled clinical trial [34]. In contrast, many so-called “alternative” modalities are satisfied to rely on a modality’s ancientness (whether real or merely assumed), weak or non-replicable studies, and/or unverifiable personal testimonies that may or may not reflect how most patients would be affected [32],[33]. In some cases, alternative practitioners persist even after a substantial clinical evidence directly contradicts the premises underlying the modality, such as is the case with homeopathy [47]. That’s not to say that there do not exist certain exceptions, but the overarching pattern is that the standards of evidence agreed upon by mainstream medical researchers is different than the standards deemed acceptable in alt med. The agreed upon standards of evidence in scientific medicine represents what is being referred to here as social calibration. A counter-example would be something like ghost hunting, whereby there do not exist any consistent standards insofar as what is supposed to qualify as evidence for a ghost [48].
Social Diversity: This criterion simply means that the evidence and analyses comprising the scientific consensus should come from varied sources by scientists of varied backgrounds and cultures in order to avoid any systematic bias in the scientific literature [1]. For example, one of the arguments against the international scientific consensus on genetically engineered food safety is based on the misconception that seed companies like Monsanto are the only ones doing the research, or that they dictate who does. That’s not actually true, but if it were, then the evidence would be falling short on this criterion. Instead, the GE food consensus is supported by myriad studies by scientists from different ethnic, cultural and economic backgrounds with varied funding sources from all around the world, and by position statements from the most prestigious scientific organizations on the planet [27],[28],[29],[30]. The Social Diversity criterion ensures that a consensus is not a product of group think, politics, financial incentives, ideological motives, or shared cultural values.
Via John Garrett of Skeptical Science for Denial 101
Scientific Consensus =/= Unanimity
Notice that the aforementioned criteria defining scientific consensus does not preclude normative contestation or outlier viewpoints within the scientific community. That’s actually quite normal and usually fairly benign. Total absolute 100% unanimity among experts is not a prerequisite to a consilience of evidence supporting a particular conclusion.
Not All Disagreements are of Equal Merit
However, there are cases in which that normative contestation and areas of scientific uncertainty becomes exaggerated by groups with ulterior (unscientific) motivations (either financial, political, or ideological) in order to argue against the reliability of extant scientific knowledge and/or to obfuscate public understanding of it. This phenomenon of special interest groups combating scientific conensus is well-documented on topics ranging from the risks of smoking tobacco, anthropogenic global warming, and the safety of GMO foods and conventional vaccine schedules [2],[3],[4],[5],[6],[7],[8],[9],[10],[11],[12],[13],[14],[15],[16],[17],[19].
With a few exceptions, however, the contrarians in these cases are attempting to shed doubt on the existence, strength, and/or legitimacy of the scientific consensus on a particular topic, and are not necessarily directly contesting the very concept of scientific consensus itself. Rather than claiming that there can be no such thing as scientific consensus, most of them instead argue that such a situation does not exist with respect to the particular topic on which they disagree, or that it may exist but is nevertheless simply unfounded. They insert false balance and exploit the normal everyday uncertainties and tentativeness inherent to all scientific knowledge and attempt to amplify them with respect to topics they wish to portray as more contentious than the evidence actually suggests. This is effective because acceptance that a scientific consensus exists has been shown to function as a “gateway belief” to accepting that a set of propositions is true, and manufacturing the appearance of continued legitimate scientific controversy can obscure public perception of its existence [20],[21].
However, there are some exceptions whereby contrarians attempt to undermine the very concept of scientific consensus itself (i.e. here). As you may have guessed, the claimants in such cases invariably misrepresent what scientific consensus actually is. They try to portray it as analogous to arriving at a conclusion by way of an opinion poll, which is an example of the fallacy of equivocation [18]. They will typically argue you that what matters is the scientific evidence, which, although true, ignores the fact that a concilience of evidence is already a non-negotiable prerequisite to scientific consensus [1]. It is therefore nonsensical to speak of evidence and scientific consensus as if the latter was not contingent upon the former, let alone to imply that they are mutually exclusive. Again, this is no more legitimate than arguing against scientific theories by equivocating to a colloquial definition of theory.
Based on the criteria described earlier (consilience of evidence, social calibration, and social diversity), it’s easy to see how scientific consensus will unavoidably emerge on any question for which repeated applications of the scientific method by a diverse group of scientists result in a body of evidence whose results lean heavily towards certain conclusions and away from others. Although it is possible to quantitatively analyze the nodes and connections of various scientific citation networks and how they evolve over time with respect to a given topic, the specific pathway by which scientific consensus emerges tends to vary from case to case.
This paper entitled The Temporal Structure of Scientific Consensus Formation (by Shwed et al) explains some methods by which such scientific citation networks can be analyzed to ascertain degrees of scientific consensus and to partition nodes into salient sub-communities [2]. There are both pros and cons to this approach, but such efforts are designed to minimize dependence on the discretion of individual scientists in the detection of scientific consensus. When a new topic of study first arises, different scientists typically end up in different camps which approach key questions differently, explore different initial hunches, and form different citation networks as more and more studies are produced. As a scientific consensus begins to form, the lines of distinction between the various camps begin to dissolve, and members of each start to converge on certain areas of agreement.
Somewhat counter-intuitively, the authors also discovered that when scientific consensus is achieved, the aforementioned scientific citation networks tend to grow in size, as does the total rate of literature output on the topic. At least, that was the case in the instances they analyzed. The relevance of this observation is that any consensus arrived at on the basis of weak or faulty evidence will tend to quickly dissolve as interest in (and scrutiny towards) the topic increases, whereas conclusions based on stronger evidence will tend to open up follow-up questions whose study results are complimentary to them. This is another key to understanding why scientific consensus represents not the death of scientific inquiry, but rather a scaffolding on which subsequent scientific inquiry can build and grow. It simply represents what we’ve learned so far about some aspect of the universe. In this way, scientific consensus is not so much a final point of arrival but rather a launching point for further inquiry to add to and refine our current knowledge. The authors also identified three trajectories along which scientific consensus emerges, which they characterize as “flat, spiral, and cyclic,” but I’ll leave that for readers compelled to read the original paper [2].
If the scientific consensus is wrong on some topic, then the subsequent exploration of additional questions derived from it should reveal that. There’s a reason why we refer to science as a self-correcting enterprise. It’s not just a catch phrase. Efforts to better understand the universe must build upon existing knowledge. Scientific consensus helps shape the discussion and guide resource allocation with respect to tangential and/or follow-up questions within a particular sub-discipline. Without it, we would simply keep spinning our wheels by re-establishing the same conclusions over and over again without ever attempting to build on that foundation and generate new knowledge.
Why Scientific Consensus Matters
Although often colored by personal values, biases, competing motives, and desires, humans generally make decisions based on what they perceive to be true. This is true both on the individual and group levels. Not everyone can be an expert in a scientific discipline, and nobody can be an expert in all of them. Consequently, people routinely have to assess what is likely to be true in areas on which they are not experts, and make decisions based on it. Scientific Consensus represents the most reliably accurate knowledge available to human beings on a given topic at any given time. It’s far from infallible, but then again, so is every other epistemological framework available to humans (albeit even more so). To reject it on the grounds that it is not infallible in favor of even less reliable approaches to knowledge would be an example of the Nirvana Fallacy [31]. The best available option, even if imperfect, is nevertheless still the best available option. Scientific consensus also serves as a launching point guiding further scientific study of related questions, and helps facilitate the generation and accumulation of new knowledge.
Detecting Scientific Consensus
I think it’s a fairly safe assumption that analyzing scientific citation networks with sophisticated algorithms and statistical methods is not something that the average person is likely to do with respect to every scientific claim they stumble upon. It’s not the be all end all anyway, because it only shows how sub-communities of scientists and their published work converges over time. It does nothing to adjudicate on the quality of individual studies within a citation network or the reliability of their conclusions. Nor does it distinguish between cases where a cited work’s findings are being used as supporting evidence, versus cases where a cited work’s conclusions are being challenged. It’s useful, but it’s not a replacement for actual human experts capable of summarizing the state of affairs in their fields of specialization. Ultimately, becoming an expert in a particular field would be the ideal way to equip oneself to competently assess the current state of the science within it, but that’s not feasible for most people, and nobody is an expert on every topic.
Systematic Reviews as Proxies
Fortunately, however, there are other proxies one can look for to get a ballpark idea of the degree of scientific consensus (or lack thereof) on a particular topic. For example, on many thoroughly studied topics there exist systematic reviews and/or meta-analyses which examine many studies at a time in order to assess what is implied by the weight of the evidence. A systematic review seeks to answer a specific research question by summarizing all the available scientific literature fitting a pre-specified set of eligibility criteria; a meta-analysis seeks to use statistical methods to summarize and analyze the results of such studies. Systematic reviews can vary widely in quality just like other types of studies. You can get an idea of what a good systematic review should entail and how to read one here [22],[23],[35],[44],[45].
Position Statements as Proxies
Alternatively, many reputable scientific organizations will put together position statements on certain topics, which can be a useful proxy for ascertaining the degree of scientific consensus that exists for a given topic. If the majority of prestigious organizations have arrived at similar conclusions, then the chances are that there is a fairly strong scientific consensus on the subject. Obviously this is an imperfect proxy, because there are also front organizations which masquerade as objective scientific organizations, but which are really serving some other agenda, and because it doesn’t provide a clear view of the evidence upon which their conclusions are based.
Other Proxies
It’s advisable to avoid relying too heavily on petitions or surveys of scientists’ opinions as a proxy for or against the existence of a scientific consensus, particularly on topics that tend to be controversial in public discourse. It’s not that they can’t ever be done in such a way that they could convey useful information, but rather that they’re too easy to screw up, or to be manipulated into conveying misleading information. In fact, that’s a common tactic used by people whose goal it is to obfuscate public understanding by disputing the existence of a scientific consensus on particular topics where it exists. They accomplish this by cobbling together signatures and/or statements from people whose views comport with theirs, but whose qualifications are often tangential to the topic under discussion, and/or whose opinions represent a tiny minority of researchers, and are not well-supported by the weight of the evidence in the peer-reviewed literature.
For example, the debunked Oregon Petition Project was an attempt to obscure the weight of the scientific consensus on human-caused climate change [24]. A document assembled by the Discovery Institute which boasted of 100+ scientists who reject the theory of evolution was humorously rebutted by the National Center for Science Education with Project Steve: a list comprised exclusively of scientists named Steve who accept evolution, which nevertheless dwarfed the Discovery Institute’s list [25]. Similarly, anti-GMO campaigners have written things such as the I-SIS letter as an attempt to sew uncertainty and doubt on the mainstream scientific consensus position on the safety of Genetically Engineered food crops [26],[27]. HIV/AIDS denialists have also attempted similar tactics [46].
One possible exception to this rule of thumb would be a survey which groups the participating scientists according to the degree to which their area of specialty pertains to the subject under discussion so that one can see whether the percentage of agreement increases the closer the areas of expertise get to the specific topic. Even then it would have to be based on a representative sample of each sub-category of scientists, and I wouldn’t recommended relying on it as anything more than a complimentary proxy with which to cross-corroborate with other signs of an extant scientific consensus.
Above all, avoid relying on unsourced YouTube conspiracy videos, opinionated people with no relevant scientific education, blogs and other websites that make sensational claims for which they don’t cite credible research, and fake experts whose claims are totally inconsistent with the peer-reviewed scientific literature.
Via Sheeple
It’s perfectly fine to use a video medium to learn about science, but just understand that there is no vetting process whatsoever, so content creators can say whatever they want with impunity. University lectures are usually fine (and recommended), as are tutorials videos such as Khan Academy, and any videos which cite credible sources in their video description. This should go without saying, but I’m including it for the sake of completeness.
Conclusion
Scientific consensus is not a part of the scientific method so much as it is a consequence of it. It inevitably arises whenever a large body of scientific literature accumulates that points towards similar conclusions. Typically, people who argue otherwise are equivocating due to them either misunderstanding or deliberately misrepresenting the meaning of the term. Scientific consensus is characterized by the co-existence of a consilience of evidence, social calibration, and social diversity, and although not infallible, nevertheless represents the best knowledge currently available on a given scientific question at a given time. Furthermore, it is instrumental to the generation and accumulation of new knowledge in that it directs researchers toward complimentary follow-up questions whose exploration allows humankind to build upon previous knowledge.
References:
[1] Miller, B. (2013). When is consensus knowledge based? Distinguishing shared knowledge from mere agreement. Synthese, 190(7), 1293-1316.
[2] Shwed, U., & Bearman, P. S. (2010). The temporal structure of scientific consensus formation. American sociological review, 75(6), 817-840.
[3] McCright, A. M., & Dunlap, R. E. (2000). Challenging global warming as a social problem: An analysis of the conservative movement’s counter-claims. Social problems, 47(4), 499-522.
[4] Proctor, R. N. (2012). The history of the discovery of the cigarette–lung cancer link: evidentiary traditions, corporate denial, global toll. Tobacco Control, 21(2), 87-91.
[5] Lopipero, P. A., & Bero, L. A. (2006). Tobacco interests or the public interest: 20 years of industry strategies to undermine airline smoking restrictions. Tobacco Control, 15(4), 323.
[6] SCHURMAN, R. (2004). Fighting Frankenfoods: Industry opportunity structures and the efficacy of the anti-biotech movement in Western Europe. Social problems, 51(2), 243-268.
[7] Wales, C., & Mythen, G. (2002). Risky discourses: the politics of GM foods. Environmental Politics, 11(2), 121-144.
[8] Anti-GMO Activists Are the One Practicing Tobacco Science. (2015). Food and Farm Discussion Lab. Retrieved 4 August 2017, from https://fafdl.org/blog/2015/05/19/anti-gmo-activists-are-the-ones-practicing-tobacco-science/
[9] Boykoffa, M. T., & Boykoffb, J. M. (2004). Balance as bias: global warming and the US prestige press$. Global Environmental Change, 14, 125-136.
[10] Dunlap, R. E., & McCright, A. M. (2008). A Widening Gap: Republican and Democratic Views on Climate Change.
[11] McCright, A. M., & Dunlap, R. E. (2011). THE POLITICIZATION OF CLIMATE CHANGE AND POLARIZATION IN THE AMERICAN PUBLIC’S VIEWS OF GLOBAL WARMING, 2001–2010tsq_1198 155.. 194. The Sociological Quarterly, 52, 155-194.
[12] Gary Ruskin, GMO Labeling Movement Funded by Anti-Vaxxers | American Council on Science and Health. (2017). Acsh.org. Retrieved 4 August 2017, from http://www.acsh.org/news/2017/05/08/gary-ruskin-gmo-labeling-movement-funded-anti-vaxxers-11245 Attach quote
[13] The Anti-Vaccine And Anti-GMO Movements Are Inextricably Linked And Cause Preventable Suffering. (2017). Forbes.com. Retrieved 4 August 2017, from https://www.forbes.com/sites/kavinsenapathy/2017/05/18/the-anti-vaccine-and-anti-gmo-movements-are-inextricably-linked-and-cause-preventable-suffering/#7c1113b536a2
[14] SCHICK, S. F., & GLANTZ, S. A. (2007). Old ways, new means: tobacco industry funding of academic and private sector scientists since the Master Settlement Agreement. Tobacco control, 16(3), 157-164.
[15] Kata, A. (2012). Anti-vaccine activists, Web 2.0, and the postmodern paradigm–an overview of tactics and tropes used online by the anti-vaccination movement. Vaccine, 30(25), 3778.
[16] Offit, P. A. (2015). Deadly choices: How the anti-vaccine movement threatens us all. Basic Books (AZ).
[17] Jolley, D., & Douglas, K. M. (2014). The effects of anti-vaccine conspiracy theories on vaccination intentions. PloS one, 9(2), e89177.
[18] Equivocation. (2017). https://www.logicallyfallacious.com. Retrieved 5 August 2017, from https://www.logicallyfallacious.com/tools/lp/Bo/LogicalFallacies/81/Equivocation
[19] Something About Pots and Kettles. (2015). The Skeptical Beard. Retrieved 5 August 2017, fromhttps://economicaltruth.wordpress.com/2015/02/16/pots-and-kettles/
[20] van der Linden, S. L., Leiserowitz, A. A., Feinberg, G. D., & Maibach, E. W. (2015). The scientific consensus on climate change as a gateway belief: Experimental evidence. PloS one, 10(2), e0118489.
[21] Lewandowsky, S., Gignac, G. E., & Vaughan, S. (2013). The pivotal role of perceived scientific consensus in acceptance of science. Nature Climate Change, 3(4), 399.
[22] Systematic reviews and meta-analyses: a step-by-step guide | www.ccace.ed.ac.uk. (2017). Ccace.ed.ac.uk. Retrieved 7 August 2017, from http://www.ccace.ed.ac.uk/research/software-resources/systematic-reviews-and-meta-analyses
[23] Uman, L. S. (2011). Systematic Reviews and Meta-Analyses. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 20(1), 57–59.
[24] 30,000 Scientists Reject Anthropogenic Climate Change?. (2016). Snopes.com. Retrieved 7 August 2017, from http://www.snopes.com/30000-scientists-reject-climate-change/
[25] Project Steve | NCSE. (2017). Ncse.com. Retrieved 7 August 2017, from https://ncse.com/project-steve
[26] Scientists Declare No Consensus on GMO Safety. (2017). I-sis.org.uk. Retrieved 7 August 2017, from http://www.i-sis.org.uk/Scientists_Declare_No_Consensus_on_GMO_Safety.php
[27] Hulk, C. (2015). The International Scientific Consensus On Genetically Engineered Food Safety. The Credible Hulk. Retrieved 7 August 2017, from http://crediblehulk.org/index.php/2015/11/22/the-international-scientific-consensus-on-genetically-engineered-food-safety/
[28] Sanchez, M. A. (2015). Conflict of interests and evidence base for GM crops food/feed safety research. Nature biotechnology, 33(2), 135-137.
[29] (GENERA), G. (2014). Source shows half of GMO research is independent | Ag Professional. Agprofessional.com. Retrieved 7 August 2017, from http://www.agprofessional.com/news/New-resource-shows-half-of-GMO-research-is-independent-272765251.html
[30] (2017). Genera.biofortified.org. Retrieved 7 August 2017, from http://genera.biofortified.org/wp/wp-content/uploads/2014/08/GENERA_beta_PR.pdf
[31] Man, F. (2016). The nirvana fallacy: An imperfect solution is often better than no solution. The Logic of Science. Retrieved 8 August 2017, from https://thelogicofscience.com/2016/06/20/the-nirvana-fallacy-an-imperfect-solution-is-often-better-than-no-solution/
[32] We Should Abandon the Concept of “Alternative Medicine”. (2015). Science-Based Medicine. Retrieved 8 August 2017, from https://sciencebasedmedicine.org/we-should-abandon-the-concept-of-alternative-medicine/
[33] Hunt, K., & Ernst, E. (2009). Evidence-based practice in British complementary and alternative medicine: double standards?. Journal of health services research & policy, 14(4), 219-223.
[34] Sibbald, B., & Roland, M. (1998). Understanding controlled trials. Why are randomised controlled trials important?. BMJ: British Medical Journal, 316(7126), 201.
[35] Greenhalgh, T. (1997). Papers that summarise other papers (systematic reviews and meta-analyses). BMJ: British Medical Journal, 315(7109), 672.
[36] Graphic proof that vaccines work (with sources) – Isabella B. – Medium. (2015). Medium. Retrieved 8 August 2017, from https://medium.com/@visualvaccines/graphic-proof-that-vaccines-work-with-sources-61c199429c8c
[37] Vaccines work. Period.. (2013). Science-Based Medicine. Retrieved 8 August 2017, from https://sciencebasedmedicine.org/vaccines-work-period/
[38] Hulk, C. (2017). No, Solar Variations Can’t Account for the Current Global Warming Trend. Here’s Why:. The Credible Hulk. Retrieved 8 August 2017, from http://crediblehulk.org/index.php/2017/01/17/no-solar-variations-cant-account-for-the-current-global-warming-trend-heres-why/
[39] Cook, J., Oreskes, N., Doran, P. T., Anderegg, W. R., Verheggen, B., Maibach, E. W., … & Nuccitelli, D. (2016). Consensus on consensus: a synthesis of consensus estimates on human-caused global warming. Environmental Research Letters, 11(4), 048002.
[40] Oreskes, N. (2004). The scientific consensus on climate change. Science, 306(5702), 1686-1686.
[41] Doran, P. T., & Zimmerman, M. K. (2009). Examining the scientific consensus on climate change. Eos, Transactions American Geophysical Union, 90(3), 22-23.
[42] →, V. (2015). Is There a Consensus about Climate Change?. Thoughtscapism. Retrieved 8 August 2017, from https://thoughtscapism.com/2015/03/13/is-there-a-consensus-about-climate-change/
[43] Comparison of Effects of Diseases and Vaccines – Canadian Immunization Guide. (2012). Web.archive.org. Retrieved 8 August 2017, from http://web.archive.org/web/20160325103229/http://www.phac-aspc.gc.ca/publicat/cig-gci/cedv-cemv-tab-eng.php
[44] Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., … Bouter, L. M. (2007). Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology, 7, 10. http://doi.org/10.1186/1471-2288-7-10
[45] Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P., … & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS medicine, 6(7), e1000100.
[46] Schüklenk, U. (2004). Professional responsibilities of biomedical scientists in public discourse. Journal of medical ethics, 30(1), 53-60.
[47] Hulk, C. (2015). Money For Nothing: Why Homeopathy Is Still Pseudoscientific Nonsense That Does Not Work. The Credible Hulk. Retrieved 13 August 2017, from http://crediblehulk.org/index.php/2015/12/02/money-for-nothing-why-homeopathy-is-still-pseudoscientific-nonsense-that-does-not-work/
[48] Ghost-Hunting Mistakes: Science and Pseudoscience in Ghost Investigations – CSI. (2016). Csicop.org. Retrieved 13 August 2017, from http://www.csicop.org/si/show/ghost-hunting_mistakes_science_and_pseudoscience_in_ghost_investigations
Via A Science Enthusiast
