Supporting Students in Evaluating Sources
One of the most difficult tasks that we have as humans is discerning whether something we hear, see, or read comes from a reputable source. In fact, with the internet and social media, it’s a struggle for all of us, but young people may need particular guidance in making this determination (Wineburg et al., 2016). Many news stories include science content gleaned from journal articles about genuine scientific studies. However, many stories contain spurious material, and it is easy to be fooled into believing incorrect information if you are not able to distinguish between well-documented research and unfounded assertions.
Given the rate at which science advances and the multitude of sources in which science articles are published, it is becoming increasingly clear that understanding the validity of published information is a skill that students need to cultivate. Below, we share a process students can use to critically evaluate sources of scientific information.
Identify a Current Controversy
One way to begin to help young people develop their ability to discern fact from conjecture or even downright falsehoods is to explore articles relating to current events. Take, for example, the notion of wearing a mask to protect yourself and others from airborne pathogens. This issue has caused a large amount of controversy over the past year, with each side of the debate sharing purportedly science-based arguments for their position.
Provide a Related Article
You and your students can bring in an article you think is a good example of a credible, or not-so-credible, publication. Recently, the article "Facemasks in the COVID-19 Era: A Health Hypothesis" by Baruch Vainshelboim was cited in a posting on the website of a small community newspaper. The writer asserts that the local supermarket no longer has the authority to enforce facemask-wearing and backs up the assertion with:
Share First Impressions
Ask students to share their first impressions about credibility and evidence to support their opinions. At first glance, the cited article looks to be quite legitimate. It is listed in PubMed Central, a free full-text archive of biomedical and life sciences journal literature at the U.S. National Institutes of Health's National Library of Medicine (NIH/NLM). This is a U.S. government website (as evidenced by the “.gov” in the URL). It was published in the medical journal Medical Hypotheses, which on the surface seems quite reasonable. The author lists his affiliation as Cardiology Division, Veterans Affairs Palo Alto Health Care System/Stanford University, which is seemingly a reputable organization that lends credibility to the article. It looks very much like other science journal articles you may have seen—there is an Abstract, an Introduction, mentions of the World Health Organization and other reputable scientific organizations, and a table that lists the potential health consequences of mask-wearing. Delving more deeply, you might notice that the article is littered with footnotes, listing 67 references!
Model and Invite Critical Analysis
However, it turns out that this article is not what the writer of the posting asserted it is. You can help students identify specific areas to examine to determine credibility.
Journal/publisher: Help students identify whether the publisher or publication is well known, and clarify the journal’s mission and purpose. Despite its official-looking source and format, this article appeared in the journal Medical Hypotheses. This journal’s purpose (according to the publisher) "is to publish interesting theoretical papers.” They go on to say that the journal will “consider radical, speculative and non-mainstream scientific ideas provided they are coherently expressed.” When you search for Medical Hypotheses on the internet, you’ll find that it is not conventionally peer-reviewed (reviewed and fact-checked by a team of experts in the field to avoid publishing misinformation). While Medical Hypotheses does have its own review system, it is “in accordance with the aims and scope of the journal, which is to publish radical new ideas” according to the editor-in-chief—meaning that it publishes work that includes plausible conjecture rather than provable fact.
Fact-check: Beyond simply exploring the publication, you can invite students to look critically at the claims and facts within the article itself. This may include both incorrect and unsubstantiated claims. In this example, you can find both, including:
- An assertion that the case fatality rate of COVID-19 is considerably less than 1%, which is backed up with a citation from March 26, 2020 (VERY early in the pandemic). While the true case fatality rate of COVID-19 is not definitely established, as of March 1, 2021, the lowest case fatality rate in the world (New Zealand) is 1.1% (ourworldindata.org).
- An indirect suggestion that wearing a mask can cause hypoxemia (oxygen deficit) without citing any studies that show mask-induced hypoxemia. Through a series of alarming pieces of information about the dangers of hypoxemia, it creates an air of urgency that seeks to frighten the reader about mask-wearing (“It is well established that acute significant deficit in O2 [hypoxemia] and increased levels of CO2 [hypercapnia] even for a few minutes can be severely harmful and lethal, while chronic hypoxemia and hypercapnia cause health deterioration, exacerbation of existing conditions, morbidity and ultimately mortality.”) While the references that point out the dangers of hypoxia are valid, there are no references that provide evidence connecting mask-wearing to hypoxia.
- The sources cited are not quoted nor are page number references provided. Reputable scientific papers cite sources and include page numbers for the citation and often provide directly-quoted material.
- Incorrect spelling (“...400,000 people showed a 13% increased morality [sic] risk among people”) and grammar issues (“As described earlier, wearing facemasks causing [sic] hypoxic and hypercapnic state that constantly challenges the normal homeostasis…”). While this can be a result of an author whose primary language is not English, most journals have editorial staff that ensure correct spelling and grammar before publication.
Revisit First Impressions
After exploring the sample article, have students share any factors that influenced them to confirm, alter, or change their original opinion about the credibility of the source. They may come away from the article with a much different opinion than when they first encountered it. Ask students about other contexts outside the field of science where this kind of critical analysis may be important or useful.
Developing a Media Literacy Mindset
Unfortunately, many people do not take the time to really examine and consider what they are reading. Often, it is simply a matter of giving something a quick look, deciding whether it looks “official” (which this certainly does), and accepting it as fact. So, what can we do to ensure that youth have the greatest ability to verify the information they consume? First, they need prompting to ensure that they have researched and corroborated their sources. Here are some helpful tips that should enable young people to be better media consumers from the Media Literacy Toolbox developed by the YouthLearn project at EDC. You can encourage young people to ask the following questions when listening to, reading, or watching stories:
Who's Talking?
- Who produced this media?
- Where are they from?
- Who are they with?
- What are their attitudes and values relative to mine?
- What’s their intent?
Who's Listening?
- Who cites this information?
- Who links to the site/work?
- What are their affiliations and points of view?
- Do I agree?
- How can I respond?
In the coming months, the ABE Program Office will be developing some tools and lesson plans that teachers can use with students specifically to examine scientific articles, both in the media and in scientific journals. Keep your eyes out for these new resources!