Building a Better Digital Library Together: Community Responses to Summon Discovery System

Editor's Notes

• #2: Following upon the Library’s official launch of Summon in September 2014 a user survey of the OCAD University community was undertaken in January – February 2015. The aim of the survey was to gauge user satisfaction with the new system over the course of the fall academic term and to obtain feedback to inform improvements to Summon as well as any initiatives to strengthen user support and promote the service. Survey participants were asked between 12 – 15 open and closed-ended questions ranging from user demographic questions to satisfaction with specific features of the Summon service. Research ethics approval was sought for the survey by OCAD’s Research Ethics Board; however, the board determined that approval was not necessary and this type of survey, “quality assurance”, was out of scope for the board. The survey was promoted in a variety of ways: Library news & social media, LibGuides, Facebook advertisements, several signs were posted throughout the Library, banner ads and survey invitation pop-ups, in information literacy sessions, and survey computer “kiosks” both in the Learning Zone and at the Library’s winter book sale. As incentives to participate candies were made available to survey participants that used the survey computer kiosks in the LZ and over the course of the book sale and survey respondents were invited to enter a draw to win one of three $50 gift cards for Above Ground Art Supplies. Of the promotional methods used, promoting the survey at the book sale was the most effective method followed by promoting the survey in information literacy sessions and kiosk set-up in the Learning Zone. $10 was spent promoting the survey on Facebook; however, it’s difficult to determine how effective this method was in generating responses or in promoting the survey. Facebook ads were targeted to an audience living in Toronto, with post-secondary education level, who listed OCAD University amongst their interests. While the FB advertisement campaign had extensive reach, displaying in the newsfeeds of 4,815 users, the click-through rate was relatively low (39) and of those click-throughs very few resulted in completed survey responses.
• #3: The survey collected a total of 105 responses (est. 2% of total population) with 96% (101) consenting and 4% (4) declining to participate in the survey. The largest number of survey responses came from undergraduate students, 65% (65), followed by library staff at 13% (13) then faculty at 9% (9). Additionally, graduate students accounted for 4% (4) of responses and staff (6) & other (3) for 9%.
• #4: 4th year students made up the largest proportion of undergraduate respondents at 38%, followed by 3rd year at 23 %, 2nd at 21%, and first year at 12%. Given that the 1st years represent the largest group of undergraduates yet the lowest number of responses to the survey; other methods of promoting future surveys to this group should be considered to increase the number of responses.
• #5: Of the undergraduate students surveyed, the largest numbers of respondents are enrolled in Illustration at 22% followed by Drawing & Painting at 17%, Environmental Design 11%, and Advertising 9%.
• #7: Art 2, Design 2, Liberal arts 5
• #8: First time 33, 2 – 10 29, 11 – 25 16, 26 – 50 6, 50+ 15
• #9: Library databases were selected as the commonly most used online tool, followed by the catalogue and Google.
• #10: 79% of users felt that Summon was either somewhat or very easy to use. Obviously, library staff found it the easiest of all to use with 75% saying it’s very easy – undergraduates and graduates reported more difficulty using Summon than others.
• #11: 88% found the results somewhat to very relevant to their research topics. Not at all 4, not very relevant 8, somewhat relevant 63, very relevant 22
• #13: Surprisingly, a very high number of users claimed to apply the filters/facets in their searches.
• #14: Most users agreed that search filters improved their search results. Most commonly mentioned filter was the date slider to narrow down results for more current articles. Not at all useful 0, not very useful 7, somewhat useful 31, very useful 26
• #17: Comments on the positive attributes ranged from the speed, and relative ease of discovering content useful to research projects to specific features of the UI including faceted display and sorting of results.
• #18: Comments on negative features of Summon included issues with icons used to differentiate content types or print / online material. Issues with Linking were prominent in the responses – in some cases an article level link cannot be built via OpenURL, in others linking may fail entirely. Duplication of results is a problem (When FoD was first indexed I noticed that all the records appeared twice). Complaints about known-item searching.
• #19: Overall, 85% of users reported being somewhat or very satisfied with Summon. The breakdown is: Very dissatisfied 5, somewhat dissatisfied 9, somewhat satisfied 50, very satisfied 31.
• #20: 87% indicted that they were somewhat or very likely to continue using Summon in their research. Very unlikely 6 (4 undergrads 1 faculty and 1 library staff member), somewhat unlikely 6, somewhat likely 30, very likely 53