Download to read offline
Uploaded byLynn Connaway
Researching Students’ Information Choices (RSIC): Determining Identity and Judging Credibility in Digital Spaces.
The document discusses a research project on students' information choices, focusing on their ability to evaluate credibility in digital spaces. It highlights the importance of media and information literacy for navigating online resources effectively and addresses the demographics of the research participants. The study aims to understand how students differentiate between various online resources and assess their credibility during research.
More Related Content
Similar to Researching Students’ Information Choices (RSIC): Determining Identity and Judging Credibility in Digital Spaces.
More from Lynn Connaway
![“It [library tour] wasn’t what do you do when you need to make a literature r...](https://cdn.slidesharecdn.com/ss_thumbnails/iflalacgreece2019connawayfinal-200219213428-thumbnail.jpg?width=640&height=640&fit=bounds)
Researching Students’ Information Choices (RSIC): Determining Identity and Judging Credibility in Digital Spaces.
- 2. VALA Melbourne •11 February 2020 Researching Students’information Choices (RSIC): Determining Identity and Judging Credibility in Digital Spaces Lynn Silipigni Connaway, Ph.D. Director of Library Trends and User Research [email protected] @LynnConnaway
- 3. Membership and Research’sshared purpose Scale and accelerate library learning, innovation, and collaboration
- 4. Research devoted exclusively tothe challenges facing libraries and archives Support for elected delegates to OCLC Regional and Global Councils; a venue for local and regional views on the issues facing libraries today Collaboration OCLC RLP and WebJunction as platforms to connect, contribute, and participate in discussions about challenges and solutions
- 5. • WebJunction andResearch Library Partnership (RLP) Community-supported programs that • Activate collaboration networks • Build and steward hands-on partnerships • Tackle shared problems • Develop and share emerging knowledge and practices Image by Gerd Altmann from Pixabay
- 6. People are connections toknowledge Peer-to-peer collaborative learning Strategic thinking to practical application Influence service design and future research Shared understanding for faster innovation Leverage network knowledge to fuel innovation
- 7. • Click toedit Master text styles • Second level • Third level • Fourth level • Fifth level • Click to edit Master text styles • Second level • Third level • Fourth level • Fifth level Photos by OCLC
- 8. Tara Cataldo, MLS Co-PrincipalInvestigator Rachael Elrod, M.Ed., MSLS Investigator Randy Graff, PhD Investigator Kailey Langer Research Assistant Lynn Silipigni Connaway, PhD Co-Principal Investigator Ixchel Faniel, PhD Co-Principal Investigator Erin Hood, MLIS Investigator Brittany Brannon, MA Research Assistant Joyce Valenza, PhD Co-Principal Investigator Amy Buhler, MSLS Principal Investigator Samuel Putnam, MLS Investigator Chris Cyr, PhD Investigator
- 10.
- 11. https://commons.wikimedia.org/wiki/File:Backlit_keyboard.jpg What really happenswhen student researchers meet a Google results page?
- 12. The Global Importance of Media& Information Literacy “Survival in knowledge societies requires that women, men, children and youth, in general, all citizens, have the competencies to purposefully navigate the flood of information, decipher media messages they come across, create and participate in media and interact online despite their race, gender, age, beliefs, ability or location. This rapid growth in technologies and media has opened up new forms of citizen engagement. Women/girls and men/boys use of social networking platforms has created a virtual second world. Meanwhile, a large number of studies show that citizens do not have the competencies to effectively exploit the opportunities provided by this virtual world and at the same time minimize the potential risks.”
- 13. When embarking onan in-depth research project: 1. Do students differentiate among different types of online resources during “point of selection”? ○ The What ○ The How ○ The Why 2. How do students determine the credibility of online resources?
- 14.
- 15. Demogra phics N = 175 90= higher education 85 = K12 53% female, 46% male, and 1% another gender 85% born in U.S. 36% first-generation college students
- 17. Expanded V&R Framework Educational Stage Definition Pre-Emerging 1 Elementaryand middle school students Pre-Emerging 2 Students in their first three years of high school/secondary school Emerging Last year high school/secondary school, vocational/technical school, and first-year undergraduate college/university students Establishing Upper division undergraduate college/university students Embedding Graduate/postgraduate students Experiencing Faculty/scholars/lifelong learners
- 19.
- 21.
- 23.
- 30.
- 31. Captain Mitch: Doessponsored text have value?
- 32. Wikipedia: Helpfulness vs.Citability. Stop the shaming?
- 33. Google Books: Theproblem with books
- 34.
- 35. • Moving beyondaboutness • Understanding of the nature of authority • Identifying potential and varied uses of information at different stages in the inquiry process • Considering scholarship as a conversation The Gaps Image: https://www.flickr.com/photos/oprior/64409980/by Owen Prior/CC BY-SA 2.0
- 36.
- 37. Thank You &Acknowledgements This project was made possible in part by the Institute of Museum and Library Services grant number LG-81-15-0155 Image: https://www.flickr.com/photos/oprior/64409980/by Owen Prior/CC BY-SA 2.0
- 38. Discussion & Questions LynnSilipigni Connaway, Ph.D. Director of Library Trends and User Research OCLC [email protected] @LynnConnaway Image: https://www.flickr.com/photos/huangce/8390725539/ by TripNotice.com/CC BY 2.0
Editor's Notes
- #2 Thank you for coming.
- #4 Current version of flyer: https://www.oclc.org/content/dam/research/publications/2019/research-update-june-sept-19.pdf
- #5 OCLC's Membership and Research Division serves the OCLC global network of 16,000+ member libraries and beyond in several key ways: OCLC Research concentrates its efforts on research devoted exclusively to the challenges that libraries and archives face. The OCLC Research Library Partnership includes some 130 research library partners who collaborate with other libraries and each other on research and related issues through working groups. [Notes added 8/27/18]
- #6 Supported thru community contributions and co-investment from OCLC Partnering with libraries as they evolve to meet 21st century challenges, providing the connections, knowledge, and resources to plan with confidence in a complex, rapidly changing ecosystem We also partner with agencies and organizations outside the library field who bring special knowledge or resources that can benefit libraries, and manage grant-funded special projects Provides opportunities to share tacit knowledge to create common practice
- #9 8
- #10 Here is our team at our most recent face-to-face meeting in April 2018.
- #12 In their often-cited study, Evaluating Information: The Cornerstone of Civic Online Reason, Wineburg, McGrew, Breakstone, & Ortega (2016) of the Stanford History Education Group (SHEG), presented middle school, high school and college students with a series of tasks designed to measure their ability to reason the quality of information they might meet online. The researchers summarized young people’s ability to evaluate information as bleak. Students were ill-prepared to reason the origins of a source and easily duped (p. 4). In a later SHEG study (Wineburg & McGrew, 2016), researchers compared the efforts of professional fact checkers with historians and first year college students in evaluating credibility of potential sources. The fact checkers pursued a more successful approach, spending far less time focused on carefully examining the original article or reading it vertically. After a quick scan of a source, the fact checkers opened new browser tabs and explored Web resources beyond the article to investigate the site’s credibility and claims, a practice the researchers described as reading laterally. In fact, “fact checkers, in short, learned most about a site by leaving it” (p. 53). A large-scale Project Information Literacy survey, How Students Engage with News: Five Takeaways for Educators, Journalists, and Librarians, explores how a diverse sample of nearly 6000 U.S. college students and a select group of high school students digitally gather information and engage with the news across media platforms (Head, Wihbey, Metaxas, MacMillan, & Cohen, 2018). The researchers described students’ attempt to discern authority as vexing. They share, “For many of today's young news consumers, engaging with news may feel like being on a scavenger hunt. Many know they need to invest the time and critical thinking to assemble, evaluate, and interpret news as it is delivered today.” From the student perspective, engagement with news, “takes a certain amount of effort. As mentioned, this reality left one student saying, ‘I spend more time trying to find an unbiased site than I do reading the news I find.’ Many students exert effort to find reliable, high-quality news about topics that have piqued their interest. Persistence is their watchword” (p. 29).
- #13 Information skills have critical importance beyond the imposed tasks of research simulations and beyond the need to complete school research assignments. Media and information literacy skills have critical global importance. In Media and Information Literacy: Policy and Strategy Guidelines for UNESCO (Grizzle et al., 2013), emphasize the importance of countries developing national policies and strategies for media and information literacies. The Guidelines highlight the importance of building of citizens’ abilities “to identify relevant, trustworthy information” (p. 46). Without MIL policies and strategies, the document warns of disparities relating not only to basic information access but to the emergence of additional disparities “between those who are able and unable to find, analyze and critically evaluate and apply information and media content for decision-making” (p. 14). UNESCO’s (2017) “Five Laws of Media and Information Literacy” framework illustrate a global strategy positioning these critical literacies as a combined set of competencies prerequisite for fostering equitable access to information and knowledge and promoting free, independent and pluralistic media and information systems. The peer-reviewed yearbook collection, Media and Information Literacy for the Sustainable Development Goals (Singh, Grizzle, Yee, & Culver, 2015) offers global context for the relevance of MIL to the United Nations’ Sustainable Development Goals (SDGs), specifically to democratic participation. Survival in knowledge societies requires that women, men, children and youth, in general, all citizens, have the competencies to purposefully navigate the flood of information, decipher media messages they come across, create and participate in media and interact online despite their race, gender, age, beliefs, ability or location. This rapid growth in technologies and media has opened up new forms of citizen engagement. Women/girls and men/boys use of social networking platforms has created a virtual second world. Meanwhile, a large number of studies show that citizens do not have the competencies to effectively exploit the opportunities provided by this virtual world and at the same time minimize the potential risks (p. 22). National and international standards and frameworks, including ACRL (2015), AASL (2018), IFLA (2011; 2017), and UNESCO (2017) clearly reinforce the importance of addressing information, media and digital literacy knowledge practices and dispositions with students. In a world where students are vexed by their information choices and their ability to discern authority is described as bleak, there is a sense an urgency about better understanding and ultimately addressing and supporting students’ point-of-need decision making for their school research projects to mitigate disparities to ensure long-term and effective civic engagement.
- #14 This urgency is the premise for the Researching Students’ Information Choices (RSIC) project, an Institute of Museum and Library Services (IMLS)-funded study, that examines and compares the judgments and perceptions of students as they select resources for science-related school inquiry projects. It seeks to answer these two broad research questions… The primary research objective is to examine how students determine the helpfulness, credibility and citability of online search results, particularly with regards to the role played by container, or resource type. Using an innovative methodology, 175 students representing five educational stages participated in task-based simulations to study their real-time selection of resources involving the impact of the Burmese Python in the Florida Everglades.
- #18 Student participants were grouped by educational stages defined by the Digital Visitors and Residents (V&R) framework (Connaway, Kitzie, Hood, & Harvey, 2017). The V&R educational stages were developed in response to Prensky’s (2001) digital natives and digital immigrants theory to identify how and why individuals engage with technology and how they get their information. Because age is not the sole factor in determining how one engages in technology and determines the credibility of information, the V&R framework provides a broader range of characteristics than the model offered by Prensky. The four V&R educational stages are 1) Emerging: last year high school/secondary school, vocational/technical school, and first-year undergraduate college/university students; 2) Establishing: upper division undergraduate college/university students; 3) Embedding: graduate/postgraduate students; and 4) Experiencing: faculty/scholars/lifelong learners. Since the RSIC study includes elementary and early years of high school/secondary school students, a new educational stage has been added – Pre-Emerging – to include 4th-11th grade/year students. There are two Pre-Emerging educational stages - Pre-Emerging 1 includes elementary and middle school students, and Pre-Emerging 2 includes students in their first three years of high school/secondary school. This division is preliminary, as these stages have not yet been extensively studied in the V&R framework. Future research may alter the specific division of these stages.
- #22 This paper focuses four different tasks (Helpful, Cite, Credible, and Container) in a simulated Google environment. The Helpful task asked participants to select the resources they considered helpful to address their research prompt. The Cite task displayed the resources that the participants selected as helpful, and asked participants to select those they considered citable. The Credible task asked participants to select, on a scale from 1 to 5, how credible they believe the resource is (with 5 being the most credible). Once again, participants were only asked about resources that they selected as helpful, and those who did not select each resource as helpful did not receive a score for the Credible task. The Container task asked participants to select the best container for the resource from eight possible choices.
- #25 Participants in higher V&R stages were more likely to select the Wikipedia resource as helpful than participants in lower stages. This also appears to be the case with the Google Books resource, though this relationship is less clear. There was a steady increase in the percentage of participants that selected the resource as helpful from the Pre-Emerging 1 stage to the Establish stage (3% to 53% respectively). At the Embedding stage, however, there was a drop to 30%. This suggests a possible inverted U shaped relationship, with participants at middle stages most likely to select the resource as helpful. Participants at higher V&R stages were less likely to select the Captain Mitch resource as helpful.
- #26 The Cite Task only asked participants about resources that they selected as helpful, meaning that those who did not select a resource as helpful automatically did not select it as citable. As such, it is likely that the task produced lower overall citability scores for each resource. With this caveat acknowledged, it appears that participants at higher V&R stages were more likely to select the Google Books resource as citable, but less likely to select the Captain Mitch and Wikipedia resources as citable. The Wikipedia relationship is especially interesting, since participants at higher V&R stages were more likely select that resource as helpful. This suggests that participants with higher levels of education have a nuanced view of Wikipedia, viewing at as helpful but not citable.
- #27 Among those that thought that the Captain Mitch resource was helpful, the credibility score declined by V&R stage.
- #28 For the Google Books resource, on the other hand, there was an increase in credibility score by cohort.
- #29 For the Wikipedia resource, the trend is less clear, with significant variation in credibility score within every cohort.
- #30 Participants at higher V&R stages were better able to correctly identify the Wikipedia resource as a website and the Google Books resource as a book than participants at lower V&R stages. There does not appear to be a relationship between V&R stage and correctly identifying the Captain Mitch resource, and overall participants struggled to correctly identify this resource as a blog.
- #31 The three resources analyzed here can be viewed to have three different levels of authority. Using collected qualitative data to examine student reasoning at different educational stages reveals a range of thinking about credibility from gut-level heuristics to the type of knowledge practices and dispositions advocated by the ACRL Framework for Information Literacy for Higher Education as well as the AASL Learner Standards, particularly the Shared Foundations—Curate, Explore and Engage. The following think-aloud quotes reveal typical responses related to the examined resources.
- #32 The primary purpose of the Captain Mitch resource is commercial, promoting airboat tours. While several of the participants in the Pre-emerging educational stages incorrectly viewed Captain Mitch as a news article (rather than a blog), others were able to discern the true purpose of the resource. One Pre-Emerging 1 student reflected the confusion of a more sophisticated researcher struggling with limited format cues, but able to clearly recognize a sponsored text. Captain Mitch is advertising his airboat tours, so it's not going to be a news story or news outlet...And it's saying it's certified with TripAdvisor and it’s advertising airboat rides, so it's not going to be a magazine because if it's a magazine about Burmese pythons,…it’s not going to be advertising airboat rides. One Emerging student thought more deeply about Captain Mitch while judging its credibility, considering the value of professional experience in informal sources: Well, looking without clicking this article, I would say a two just because it’s captainmitch.com and that's super weird. Airboat tours? I don’t know. Since he does airboat tours, he probably knows more. So I’ll give that a three. Just because I know that my biological father did a lot of riverboat tours and stuff, and knew a lot of stuff through doing that, which is why I [figured he would know more]. This reflection echoes the knowledge practices and dispositions of the ACRL Framework that authority is constructed and contextual, defining different types of authority and recognizing that authoritative content may be packaged formally or informally and demonstrates the potential for students to cross this important threshold concept at an earlier stage in the academic career. The student also reflects consideration of the AASL Learners Standard: Engage in their nuanced effort to evaluate “information for accuracy, validity, social and cultural context, and appropriateness for need.”
- #33 The Wikipedia website article on Burmese Pythons is, perhaps, more ambiguous in its fundamental authority. The crowd-sourced authorship of Wikipedia articles provokes debate relating to the credibility of the resource, despite the fact that many Wikipedia authors are topic experts and that librarians often bolster Wikipedia articles by editing and adding references. The university professor on the study’s advisory panel stated, I have high confidence in the material because I know that the people who contribute (I'm one, as are my students) are often quite well versed in the subject. Most importantly, the changes are transparent - I can look through the history to read the discussion, much the way I can in open peer reviews. While many students judge the helpfulness and citability of a resource at the individual resource level, it appears that credibility judgments often are made at the source level. This may explain the high rate of the Wikipedia resource’s helpfulness but drop in citability and credibility. The experts on the study’s advisory panel were more likely to judge the Wikipedia resource at the resource level, taking the time to note the references in the article and their quality. At the earlier educational stages, student decision making focused on the notion of aboutness across their range of choices. Credibility did not play a large role in their think-aloud discussions and decisions regarding helpfulness. Wikipedia appears to be an exception. While the Wikipedia article might have been judged as helpful, and was by students in the more advanced educational stages, the students in the lower educational stages saw its use as a good/bad binary choice—a choice that parents and teachers advised them to avoid. At the advanced educational stages, the adult participants echoed behaviors described in previous studies. Head & Eisenberg (2010) discussed college students’ habits of using Wikipedia in combination with other information resources and preference for its “mixture of coverage, currency, convenience, and comprehensibility in a world where credibility is less of a given or an expectation from today’s students.” The evolution of trust In Wikipedia was documented by Mothe & Sahut (2018) who noted a contrast “between a higher frequency of use of Wikipedia and a lower rate of citation.” Students in their study related positive experiences with the collaborative encyclopedia for closed and open questions. At the Emerging educational stage (community college), several participants described what might best be described as Wikipedia shaming. This also was a prevalent comment from the V&R participants, which is referred to as the Learning Black Market since the students admitted using Wikipedia for the academic work but would not cite the resource (Connaway, Lanclos, & Hood, 2013a; 2013b; White, 2011). Among the typical responses describing Wikipedia shaming: I hate that it’s shamed, that you shouldn't use it because it’s-- I don’t know…And in my generation, everything--I feel like they’re like, “They’re going to go in there.”…I’ve actually found it to be really concise and kind of like a little nugget of information that allows me to explore further. And poor Wikipedia, and I'm upset that people are mean about it [laughter]. I wouldn’t use Wikipedia just because I know it’s not scholarly. Now, me personally, I would use it if I was talking to a friend of mine. I would tell them to check out this page just because of the references. And, personally, I think that Wikipedia kind of gets downed a little bit, but I know that most colleges don’t, and I don't think my teacher would enjoy that, so I would not use Wikipedia in my report. Other Emerging stage students described how important Wikipedia was for quickly developing critical context and vocabulary compared to using more academic texts: Again, I really like concise nuggets of information. This just feels like word vomit all over the screen. I mean, not that it's not helpful. There’s big words in it that seem important, so I feel like I would be [laughter]--I don’t know. Yeah. I don’t know. It almost seems messy to have to do all this when you can just depend on Wikipedia. While this study focused solely on an imposed academic research query and did not distinguish information uses, on the whole, the Pre-Emerging participants demonstrated aversion to all uses of Wikipedia because teachers and parents clearly warned them to avoid it. Like the university professor on the study’s Advisory Panel, in the adult sample, students highly valued Wikipedia, considering it a reliable old friend for its ability to get them up to speed on an unfamiliar topic and for its references to both primary and secondary sources. At the Embedding educational stage (graduate level), students demonstrated a disposition toward evaluating the resource at the article level: Now I’m going back to the Wikipedia article because that provides me with a lot of references of authority. And I’m going to go deep into those references to see if they are helpful. That will also help determine if the Wikipedia article itself is helpful or not. Because if the references here are good, the article itself should be good. While all participants could identify Wikipedia as a pervasive presence in the information landscape, not all were able to identify its format. Is it a blog if it is open to contributions? A traditional encyclopedia would be labeled a book. Eighty-one percent of the students labeled Wikipedia as a website. The next most popular container (at 8%) was blog which was chosen more by the Pre-Emerging 1 students.
- #34 At the higher authority level is the book discovered in Google Books. The book on pythons was written by expert herpetologists and published by reputable university press in the United States. While most students were able to identify the Google Book as a book, other e-books on the simulated SERPs confounded them. In this case, the word “book” occurred all over the resource page. In other e-book choices, students were not able to articulate typical container characteristics, like publisher, chapters, tables of contents. All of the Establishing and Embedding students identified the Google Book as a book. Pre-Emerging 1 students, 20% of the elementary students and 10% of middle school students in the Pre-emerging educational level, labeled it a website. “Oh, well. I think pretty much it’s a book because it says books, and it’s obviously a book because it has all the authors and then invasive pythons in the United States and all the--it has page numbers and everything and all that, so. And pictures in [the-].” Students rarely got to the level of determining the authority of the book itself, perceiving the book format itself as too long, too old, or inaccessible as illustrated in the comments below: “This is a book to buy, not a resource I can use right now” “Invasive Pythons in the US, this is a Google book, so I’d probably have to buy it.” “Oh, but here’s that guy Dorcas again. But again, the books, I kind of--if I were looking for something specific, a lot of times it's like, “Oh, it's in this book,” and then I’ll go and actually do exactly this. I’ll open it up in the Google Books or whatever, and hopefully, the free preview lets me see the one page where I just need the citation, and then I’ll go and find that paper somewhere else just because a lot of times--and especially with books, a lot of times there [sic] older. So I’d much rather just have the journal article where they’re getting their information from.” “Google Books. Invasive pythons. Ecology of a Predator. Old textbook. Somewhat worrisome because can be out of date. They can’t very easily be updated.”
- #35 The observations reported in this paper reflect a variety of needs that can inform the design of scalable instructional models to support critical student inquiry skills as well as updated professional development strategies for teachers. The observations support evidence-based decision making for the implementation of more nuanced, iterative and sophisticated information literacy instruction grounded in frameworks, guidelines and standards.
- #36 At a variety of educational stages, student participants revealed serious gaps in knowledge practices and dispositions. These include… The agility to move beyond aboutness in assessing the value of results Understanding of the contextual nature of authority The ability to identify potential and varied uses of information at different stages in the inquiry process Consideration of scholarship as a conversation through such methods as reviewing references at the end of Wikipedia articles and considering the value of citation chaining in academic work. Image: https://www.flickr.com/photos/oprior/64409980/by Owen Prior/CC BY-SA 2.0
- #37 More importantly, instructional models need to be designed based on individual’s cognitive abilities and subject knowledge of the discipline being investigated, as demonstrated by the V&R educational stages applied in this study. Participants in the higher educational stages were more likely to identify the container for the Wikipedia and Google Books resources. Participants at higher V&R stages also were more likely to select the Wikipedia and Google Books resources as helpful, and less likely to select the Captain Mitch resource as helpful. Those students earlier in their educational stages viewed Wikipedia as a resource that parents and teachers instructed them to avoid. While those in the later educational stages find Wikipedia articles as valuable and trusted resources but indicate that they would not cite them for their academic work. This window into students’ behaviour presents an opportunity for librarians and teachers to scaffold information literacy based on students’ needs at specific times in their educational stages and to build on these literacies as they progress through their educational stages. Librarians and classroom teachers might consider leveling up through the earlier introduction and practice of critical threshold concepts as well as the vocabulary and digital cues relating to the publishing process and information formats. This paper supports the need for targeted media/information literacy models as articulated by the UNESCO, ACRL, AASL, and IFLA in their national and international standards and frameworks. Regardless of educational stages, students in elementary school through graduate school will benefit from learning ways to determine helpful, credible, and citable resources for their academic work.
- #38 Image: https://www.flickr.com/photos/thaths/4314445838/ by thaths/(CC BY-NC 2.0)
- #39 Image: https://www.flickr.com/photos/huangce/8390725539/ by TripNotice.com/CC BY 2.0