Building on students’ prior knowledge to level up their media literacy

by Haruko Yamauchi
Library Department

Why help students think about the internet?

Even before the internet, misinformation, disinformation, and good-faith but poorly reasoned arguments existed in the world. Now, of course, the speed at which flimsy or dangerous information gets disseminated and absorbed is incredible, and we know that many of our students are susceptible to taking in information from unscrutinized websites at face value.

As professors at an institution with a progressive history and mission, I believe that many of us find it healthy for students to question authority, and yet we still might despair for student researchers to put Pulitzer Prize-winning investigative journalism and an inflammatory video from a YouTube conspiracy theorist on equal footing as sources to support their theses. How can we help students balance thoughtful skepticism of traditional authority with a bit of belief that intelligent, good-faith discourse still exists beyond the academy walls, that it is worth their time to seek out rational arguments and strong evidence, and that learning to discern journalistic reputation or other markers of (fallibly human, not immune from critique, but still highly qualified) authority gives them more power as readers and thinkers?

Students often come to the library knowing they must use “reliable” sources for their research assignment, while being at a loss to explain what they think their professor meant by reliable. Sometimes they have been given shortcuts, such as a pre-approved list of sources, or a mandate to use peer-reviewed journals as the only legitimate sources of information, or a prohibition against the use of any “website” whatsoever (we librarians must then persuade them that an electronic version of a monograph in the library’s collection is most likely not what was meant by “website”).

Such restrictions may serve a particular academic purpose, and may be entirely appropriate for a given subject of inquiry (I would also like to remind everyone that the library has many academic sources that are not peer-reviewed articles…a discussion for another time!). However, as well-meaning as such restrictions are, they do not further the goal of helping students develop as independent and critical readers of the digital information that surrounds them and will continue to surround them for as long as the internet exists. Students who learn to evaluate the trustworthiness of potential sources found in the online wild, not only for their academic work, but in response to their life’s questions and needs, will be better able to participate as members of our democracy and as advocates for their own often under-represented and oppressed communities. To avoid engaging with the internet altogether as if it were a forbidden toxic waste dump will only give students the false impression that we faculty are so stuck in the twentieth century (and that we of course would never use Google for our own information needs) that we are unable to offer them any guidance in these matters, so that when it comes to navigating through the most dominant mode of information dissemination in the world, they are on their own.

Over the past few years, my library colleagues and I have worked with numerous other professors on developing approaches to help students build upon their existing knowledge to fortify critical habits of mind when evaluating resources they discover outside of faculty-curated lists or fences delineating “safe” forms of information. I describe one approach here, which I have used in several in-class research workshops to support course assignments.

Reflecting on existing habits and considering new questions

When students enter the classroom, the “do now” informal writing prompt consists of two questions: (1) When you use Google, how do you decide which results to click on? (2) When you read a website, how do you decide how much you will trust it?

Asking students these open-ended, no-one-right-answer questions gets them to activate their prior knowledge and reflect on their own habits, whether they have used the internet for any college assignment or not. It also casually but deliberately defines trust as something that might exist on a Likert scale rather than flicking on and off in simple binary reaction to “good” or “bad” websites (or “fake” or “true” news). I ask students to share their responses with the class, and most fall into a few categories, e.g., I click on the first link; I click on reliable sources; I click on things I’ve heard of before; I click on .edu, .gov., or .org sites; I click on what looks relevant; I click on facts and not opinion; I avoid clicking on ads.

As I acknowledge and chart students’ responses, I ask questions to get them to reflect further. Why click on the first link? Some answer that they don’t want to bother looking further, but many believe that Google has placed the first link on top because it is the most popular (and by a transitive property therefore most reliable) or the most relevant website for their question. Although we don’t have time for a deep dive into theories about Google’s algorithms, students are often intrigued to learn that search engine optimization exists as an enormous industry, precisely because there is so much money to be gained by gaming Google’s system. I seed our later conversations by asking what makes something look reliable or relevant to them, and raise the idea that while any well-reasoned opinion should be backed up by facts, facts alone mean nothing until they are contextualized and interpreted.

I address head-on the well-meaning but usually unhelpful idea, often learned in middle or high school, that all websites with a .org, .edu, or .gov domain are inherently more worthy than .coms. I acknowledge that this heuristic might have been an okay place to start, but now in college they can think in a more complex way. I ask them to consider that no one writes something by accident (as they know well, writing takes time and effort), and that every website is built by authors (whether individual or institutional) for a purpose. What can the domain alone tell us about any given site’s purpose? A little bit—a .com does tend to have a profit motive—so as most newspapers’ websites are .com, should students never trust anything they read in any newspaper? Is the only purpose of a newspaper or news magazine to turn a profit? Does the existence of a for-profit model invalidate everything a publication or a company might say, or can it be understood as one aspect of the website’s purpose that a thoughtful researcher keeps in mind while reading?

As for .edu (including religious colleges and for-profit colleges, the purpose of the institution may be to educate, but what, I ask students, is the purpose of their website? Students often recognize that at least one purpose is student recruitment and presenting the best face to boards, donors, and peer institutions; in 2011, Penn State’s website was resolutely silent on the legal and moral charges against coach Jerry Sandusky, as opposed to the many articles in the Philadelphia Inquirer—but students taught to look only to the .edu or .com of a website would be primed to dismiss the paper as a valid source of information.

Contrary to popular belief, the .org domain is unrestricted, but even if for the sake of argument we accept that most .orgs belong to non-profit groups advocating for a particular view or social change, I ask students what they can tell about the association’s purpose from the domain alone? A dot org might belong to a public health association, a religious sect, a right-wing social media network, or a hate group. Even if you agree with the aims of the group, the domain itself does not guarantee well-reasoned arguments or responsibly sourced information.

We briefly touch upon the ideas that .gov websites follow the purpose of the agency (the Drug Enforcement Agency and the National Institutes of Health provide different kinds of information about drugs) and that as policies and administrations change, so does the content of .gov websites, as was made dramatically clear in early 2017.

The point of this class discussion is not to discredit all .edu, .org, or .gov sites, but to raise students’ awareness that relying on domains is no substitute for critically examining the purpose, point of view, and agenda of any given site. Within this discussion of purpose, we also address the idea that many students received in middle or high school that only facts matter and “opinions” are inherently unreliable. This well-meaning principle hamstrings students in their search for worthwhile sources, who often interpret this rule to mean that any sign of a persuasive purpose or point of view is enough to disqualify a text. I have seen students dismiss out of hand thoughtful op-eds written by scholarly experts because they detected an “opinion”, and the overly simplistic formula of facts=good and opinion=bad means that some students will give more weight to a Buzzfeed-style listicle of questionably-sourced statistics than an Economist or Atlantic article that explains the important impacts of a political trend.

As an alternative to a checklist of characteristics to look for, I propose that students start with six open-ended questions, and frame their responsibility as researchers as one of critical investigation. The six questions are:

Three questions about the author(s) of the text (here meaning website, but could be any article, document, or media object):

1. Who made this? What is their point of view?
2. Qualifications: What kind of expertise or knowledge do the authors have about the topic at hand? Why do they say we should believe them?
3. Reputation: What do other people say about how trustworthy the authors are or are not?

Three questions about the text:

1. Purpose: Why are they telling us this?
2. Credibility: How well-reasoned and logical is their argument?
3. Evidence: What kind of evidence is given, and where is it from? Do I trust those sources?

Based on their responses to these questions, students must make two evaluations: How much they trust each text to be reliable, and how relevant they find the text to their particular research question.

A short play!

To take the class from abstract principles to practical, concrete application, I ask for three volunteers to act out (from their seats) a very short, relatable, and humorous play that touches on students’ prior knowledge of the subway system. The exercise sets the stage for the class to apply the above questions, which will later be used to examine websites.

Each actor has a short script (which the entire class can follow along on their worksheets), and they all introduce themselves out loud: person (A) has a master’s degree in urban transportation and many years of work experience with the MTA; person (B) is a native New Yorker who takes the 4 train every day; and person (C) is a tourist who has spent exactly one day on the subway.

The question I pose to the characters is how the MTA, upon receipt of a windfall from the state, should decide which train line should receive the largest increase of trains during rush hour. Person (A) advises that the MTA base its decision on ridership numbers, cites an independent study that confirms MTA data that the L line has had the largest increase in ridership, refers the audience to a website where they can examine the data, and argues that the L should therefore get the highest increase. Person (B) argues in passionate language that “the 4 is the worst!” and nearly impossible to board on the first attempt at rush hour, that two other trains that s/he used to take were never so bad, and that anyone who says any other line should get an increase is clearly an idiot. Person (C) recounts his/her crowded trip on the A, and says that a friend who used to live in New York once saw a woman go into labor on an A platform, so clearly the A is the craziest line and should get more trains.

I then facilitate class discussion as students answer the questions above regarding each “author” and the substance of their “text”, in this case their argument for how the MTA should spend its money (excepting that in this example we can’t research the reputations of hypothetical characters, and they all share a common purpose). Students are lively in their responses, and the conversation gives us the chance to examine:

• how our own biases tend toward opinions that mirror our own (all uniformly agree that the 4 train rider is certainly not lying);
• whether or not an honestly held opinion based on extensive personal experience makes for a persuasive argument about what an entire system should do (I note that for a different question, say, “What would my commute to Manhattan be like if I moved to Burnside and Jerome?” person (B)’s personal testimony could indeed be a relevant and persuasive source);
• the relative flimsiness of arguments based on hearsay and outlier anecdotes;
• how much an institutional credential is worth—many students accept the work experience of person (A) as an obvious qualification, but those with a deep distrust of the MTA question the person’s motives (one student even suggested that they might be looking for kickbacks);
• what kinds of questions we should ask when a text offers up statistics: in this case, person (A)’s citing of a source we could examine ourselves is reassuring to most, while some students have astutely asked questions such as: “but if they’re only counting swipes, what about all the people who jump the turnstiles?”, allowing us to consider the idea that critical readers should pay attention to research methodology.

This simple exercise encourages students to continue to ask questions as they go on to search for websites related to their research assignment. After five minutes of students’ searching, while the course professor and I circulate, I ask for a volunteer to offer a website that the entire class then examines using the six questions above (the questions are also expanded upon slightly in their worksheet, for students’ future reference). In the past I would provide sample websites that I believed would raise interesting questions and asked students to evaluate them (it was in such workshops that I found out how often students dismiss rational and evidence-supported op-eds because they express an opinion). However, having students choose which website to evaluate is a more authentic experience and unearths the kinds of questions they are likely to encounter as researchers.

Student reflections

On the back of the post-it on which students wrote about how they decide which websites to click on and how they decide how much to trust each one, I ask them to tell me, anonymously, if they will use anything we’ve done in class the next time they are researching online, and if so, what. Asking them to put their reflection into words encourages some metacognitive reflection about what they have learned, and having them use the same piece of paper allows me to compare pre- and post-session answers. Some examples of paired responses include (all comments below are verbatim, save for minor spelling and punctuation corrections):

In earlier workshops this fall that did not include the pre-class writing prompt, end-of-class reflections included:

• One thing that we talked about in class today that I never really thought about was analyzing a website. I don’t always think about that, now I will.
• Researching the website itself while doing research helps you feel confident in the information you’re using and knowing what you’re saying.
• .gov .edu and .org are not the only credible domain. You must read and come to a conclusion on if the site can be trusted or not.
• Checking qualification and reliable sources is very important. Also opinions can be important and reliable as well.
• The information we received today now sets a higher standard on how to research and dig around for credibility.

It’s still all hit and miss. A lot of student feedback is vague, and interactions throughout class indicate that lessons like this will be received and integrated into research habits to very different extents by different students. I also realize that the questions about author and text are pared down and foundational, and can only start to highlight the importance of considering the source and questioning an argument’s strength and evidence. Single-class workshops certainly cannot be comprehensive training in the critical reading of sources (I would here like to promote the upcoming capstone course in information literacy!) and no one course, let alone one workshop, can fully address the need for contextual knowledge that enables students to make well-informed judgments about their sources. However, I believe these fundamental questions and habits of thinking are important to share with students, and look forward to working more with course professors to integrate the evaluation of websites into assignments, to help students develop their critical thinking skills as researchers.