Start Submission Become a Reviewer

Reading: Recommendations Without Results: What We Learned About Our Organization Through Subject Guid...

Download

A- A+
Alt. Display

Case Studies

Recommendations Without Results: What We Learned About Our Organization Through Subject Guide Usability Studies

Authors:

Sarah Gardner,

Carahsoft, US
X close

Hillary Ostermiller,

Columbia College Chicago, US
X close

Elizabeth Price ,

James Madison University, US
X close

David Vess,

James Madison University, US
X close

Alyssa Young

James Madison University, US
X close

Abstract

Many academic librarians spend significant time creating curated lists of databases, library materials, and other resources relevant to a discipline, often called subject guides. This case study examines the experience of a small team of librarians who conducted usability studies on James Madison University Libraries’ subject guides. We identified areas where our guides needed improvement and provided a list of recommendations to our liaison colleagues. While two of our key recommendations were adopted, others languished because we misconstrued our ability to initiate change beyond our small project team. Specifically, we underestimated how the absence of a content strategy for our subject guides and a lack of clear authority for creating or enforcing one would hinder our efforts. Other libraries might be able to learn from our experiences, especially from the mistakes we made.

How to Cite: Gardner, S., Ostermiller, H., Price, E., Vess, D. and Young, A., 2021. Recommendations Without Results: What We Learned About Our Organization Through Subject Guide Usability Studies. Virginia Libraries, 65(1), p.6. DOI: http://doi.org/10.21061/valib.v65i1.624
104
Views
8
Downloads
3
Twitter
  Published on 10 Nov 2021
 Accepted on 05 Oct 2021            Submitted on 11 Sep 2021

Introduction

James Madison University (JMU) Libraries offers an array of services to student and faculty users to assist in academic endeavors. One research tool offered, subject guides, provides students with a curated list of essential databases, library materials, and other resources relevant to a subject area that can be used to complete assignments and creative projects. These guides are often presented in a “pathfinder” format with resources on a topic or within a discipline grouped by type such as books, articles, websites, etc.1 More than 6,000 libraries worldwide, including JMU Libraries, use Springshare’s LibGuides software to create these pathfinders.2 Three liaison librarians new to JMU wanted to know more about how well the subject guides they inherited were—or were not—meeting their students’ needs. Before revamping them, the research team partnered with JMU’s user experience (UX) librarian and an undergraduate researcher to conduct usability studies to identify gaps between librarians’ perceptions of effective guides and students’ ability to successfully use them.

Usability studies enable organizations to evaluate a product or service to identify frustrations or problems.3 The goals of our research project were adapted from similar studies4 and aimed to:

  • Determine users’ ability to find online subject guides.
  • Evaluate whether navigation labels are clear to users.
  • Determine whether there is irrelevant content or missing instructions.
  • Evaluate how users connect to additional research assistance.

The research team carried out two rounds of usability studies between fall 2017 and spring 2019. As expected, the team identified issues with students’ ability to find and use subject guides. More importantly, the team also identified a gap in our organization’s management of the guides and our website content strategy. This second gap prevented the team from addressing the initial problem they sought to resolve and surfaced issues that might be beneficial for other librarians to identify before carrying out usability studies in their organizations.

Background

When we conceived the usability study project in 2017, all JMU Libraries subject guides were using tabbed view navigation and were organized by source type, aka the pathfinder style5 (Figure 1). Other shared characteristics included database descriptions relying on standard vendor-provided language and profile boxes located on the right-hand side of the guide. Each librarian had full authority to tailor their guides however they wished, meaning consistency across guides was non-existent. Box titles, fonts, a system-level search box on the subject landing page, and a Search This Guide box on all individual pages were beyond the guide author’s control. Those items were maintained by our web design team. We ran a pilot study with five JMU Libraries student workers as the participants in November 2017 before undertaking the formal studies with seven undergraduate students during spring 2018 and spring 2019.

Figure 1 

Example of Biology Guide during usability studies, featuring tabbed-top navigation, the profile box on the right side, and the “Search this Guide” box at top right.

What We Learned and What We Recommended

From the usability study, we learned that our subject guides are not always successful as standalone learning objects. Our recommendations focused mainly on how guides are organized, their formatting, and their content. In short, we advocated for:

  1. Renaming them “Research Guides” to match how students described them.
  2. Disabling the search boxes on the subject landing page and individual guide pages since participants clearly did not understand what content would be searched.
  3. Eliminating search widgets for the catalog and other online tools. Instead, we suggested creating links for those tools and making the link text more descriptive (e.g., “Search the Library Catalog for Books”).
  4. Moving universally to side navigation because of clear confusion about the tabbed navigation atop pages observed in our study and supported by the findings of previous studies.6 With this move, profile boxes would appear on the left side of the page on all guides. Both of these moves would reduce inconsistencies across guides and increase interchangeability that users expected.
  5. Reducing the amount of text on the guides and adding headings to all boxes to assist navigation, since the small font on the box labels were not prominent to users (this styling decision was beyond our control to fix organization-wide).

We were successful in lobbying for the guides to be renamed and used the UX librarian’s administrator access to disable the search boxes. We applied the other suggestions to our own guides (Figure 2), then we distilled these recommendations into a series of weekly email tips shared with our liaison colleagues in summer 2018 (Appendix). We hoped this informal dissemination of design tips might enable our colleagues to make some small, user-friendly tweaks to their guides before we undertook a wholesale redesign project as an organization.

Figure 2 

Example of revised Biology Guide, using side navigation, the profile box on the left, and search boxes suppressed.

Where We Ran into Barriers

Based on our initial recommendations, a few colleagues made changes to their guides, such as reducing the number of search widgets and changing some guides to side navigation. In hindsight, our one-way communication clearly lacked buy-in from our peers. We appealed to our departmental leadership to prioritize our users’ experience by mandating changes across the organization. That turned out to be a far from straightforward request for a few reasons.

At the heart of this issue, we felt, was a lack of consensus about the purpose and role of subject guides on our campus and murkiness about who owns LibGuides in our organization. Authors view guides as individual artifacts rather than as a collection of virtual service points. While our liaison department was ultimately considered the main coordinating unit for LibGuides, other units had coordinating roles for different Springshare tools and the main library website. These included the Database A–Z list that determines database descriptions and other Assets (collections unit), the LibAnswers and FAQ features (public services unit), and the custom CSS code and subject landing pages (web and application services unit). Our organization’s small usability team collaborated across these units, and LibGuides administrators came from all of these areas. Coordination was handled informally among key individuals or the directors in those units, but they had many other responsibilities in their portfolios. No individual or unit had ultimate decision-making authority. That remains the case today.

The other significant problem was that we entered our study with the mistaken assumption that as the largest group of subject guide creators (our department owns more than 90%), the liaison librarians had more authority to make recommendations about guide style and our content strategy. In our project scope, we also failed to account for broader implementation of our recommendations, a mistake shared equally by the three of us, who were newcomers in the organization, and our first-year, interim department head. The interim director was supportive of our efforts but limited in how strongly she could advocate because of the tenuousness of her own position. She had originally accepted a one-year leadership position that became indefinite amid broader leadership changes in our organization. Furthermore, our project team expressed concerns about having the director’s position be the de facto coordinator/administrator of subject guides when that job’s responsibilities already exceeded a reasonable capacity for a unit head supervising thirteen faculty members and serving as a liaison herself.

Feeling that our initial efforts stalled because of a leadership void, we pivoted and recommended that our organization create a position to coordinate the subject guides and eliminate ambiguity. We argued that having this position would allow us to not only coordinate across stakeholders, but also to develop and implement an organizational strategy for our subject guides. The coordinator would lead a standing committee empowered to develop a style guide and a process for systematically reviewing and refreshing content as recommended by other studies.7 Situating these duties within a dedicated coordinator position would make it easier to repeat usability studies cyclically to ensure our guides evolve with our users’ needs and that we retire low-use guides. This position would be responsible for maintaining storage guides, updating widgets, and removing unassigned assets from the system. A graduate assistant has been assisting our department’s director with some of these tasks, but we would benefit from having a coordinator who is empowered to notify authors about style or functionality changes and ensure they are implemented smoothly across the organization. Reflecting on our organization’s difficulty in adding new positions, we believe this role could be crafted as a multi-year service opportunity that would enable the successful candidate to gain leadership experience while demonstrating significant impact and innovation in our organization.

This type of coordination position has understandably not been a priority since we began examining the usability of our subject guides in 2017–18. We have experienced significant change among our organization’s leadership team, hiring a new dean in 2019 and a new associate dean in 2020. We have further reorganized and changed leadership in several of the other units who are stakeholders in subject guides. For instance, the interim director of the liaison unit has been hired permanently into that role; the UX team has been disbanded; the web team has overseen a website redesign project; and a new Communication & Outreach department has formed to help market our services. Now a majority of our liaison colleagues have moved to side navigation and we are using fewer search widgets. But this progress required significant invisible labor from our department’s interim director over the last three years to have conversations with individuals, get their buy-in to the suggested changes, and work on writing LibGuides revisions into their annual goals. Additional successes include the formation of a LibApps Coordinator group, documentation about the different LibApps systems and stakeholders, and drafting of a local LibGuides Content Policy. The policy has been on hold while the web team works on developing a similar content policy for the main JMU Libraries website.

The COVID-19 pandemic made online service points like subject guides more critical than ever. JMU Libraries is not an outlier in struggling to corral subject guide management. Many libraries do not create or enforce standards when it comes to subject guides. Darcy Del Bosque and Sara E. Morris reported that LibGuides administration is often unclear in organizations, with almost half of content creators unclear how many administrators were in their system and a lack of clarity about what standards were applied or enforced.8 As they noted, “Frequently answers about standards were tempered with, ‘This is encouraged but often overlooked,’ ‘It has been challenging to enforce,’ or ‘This has been hit or miss.’9 Furthermore, only 30% of content creators said their libraries had plans for retiring unneeded guides10 In another study, Judith Logan and Michelle Spence reported that only 53% of institutions had content guidelines for their guides, only 27% reviewed guides systematically, and only 26% performed user studies on them.11 This highlights a need for libraries to embrace content strategy for their websites, social media accounts, and subject guides. Courtney McDonald and Heidi Burkhardt described content strategy as “activities related to creating, updating, and managing content that is intentional, useful, usable, well-structured, easily found, and easily understood, all while supporting an organization’s strategic goals.”12 As JMU Libraries re-engages with sidelined initiatives like content policies for both our subject guides main we hope to see our organization make what McDonald and Burkhardt describe as a necessary “cultural shift toward a more collective, collaborative model of web content management and governance” that benefits our users.13

Conclusion

We set out in 2017 to conduct usability studies on the JMU Libraries subject guides that would lead to more user-centered content across our organization. From these usability studies, we made five key recommendations: (1) renaming them as “Research Guides”; (2) disabling the search boxes; (3) eliminating search widgets for the catalog and databases; (4) moving universally to side navigation; and (5) reducing the amount of text and using rich text headings instead of box labels to aid user navigation. While the first two recommendations were adopted, others languished because we misjudged our ability to initiate change beyond our small project team. Specifically, we underestimated how the absence of a content strategy for our subject guides and a lack of clear authority for creating or enforcing one would hinder our efforts. Based on the literature, other libraries are struggling with these problems, too. We believe that our experience might be a cautionary tale for librarians pursuing similar projects. Future teams carrying out usability studies should be sure to start the project with realistic and implementable outcomes, to include representatives from all relevant library stakeholders, and to have a clear charge from administration empowering them to enact their recommendations.

Discussion Questions

  1. Are subject guides a “service point” in libraries? Why or why not?
  2. What assumptions did the research team make at the outset of their project? How could they and their supervisor have set the project scope differently?
  3. How does unclear responsibility for a product like subject guides affect library employees? Library supervisors? Library users?
  4. What are the advantages and disadvantages of a content strategy for library websites? Should a content strategy extend to subject guides? Why or why not?

Additional File

The additional file for this article can be found as follows:

Appendix

JMU Libraries LibGuides Tips of the Week. DOI: https://doi.org/10.21061/valib.v65i1.624.s1

Notes

1Yoo Young Lee and M. Sara Lowe, “Building Positive Learning Experiences through Pedagogical Research Guide Design,” Journal of Web Librarianship 12, no. 4 (October 2, 2018): 205–31, https://doi.org/10.1080/19322909.2018.1499453. 

2“OCLC Transfers QuestionPoint to Springshare,” OCLC, May 31, 2019, https://www.oclc.org/en/news/releases/2019/20190531-oclc-transfers-questionpoint-to-springshare.html. 

3Department of Health and Human Services, “Planning a Usability Test” (Department of Health and Human Services, September 18, 2013), https://www.usability.gov/how-to-and-tools/methods/planning-usability-testing.html. 

4Christine Tawatao et al., “LibGuides Usability Testing: Customizing a Product to Work for Your Users,” n.d., 17., Kumar and Farney, “Planning, Implementation, and Beyond.” 

5Lee and Lowe, “Building Positive Learning Experiences through Pedagogical Research Guide Design.” 

6Jenny Corbin and Sharon Karasmanis, “Health Sciences Information Literacy Modules Usability Testing Report,” 2009, 46; Tawatao et al., “LibGuides Usability Testing: Customizing a Product to Work for Your Users”; Dana Ouellette, “Les Guides Par Sujets Dans Ies Bibiiothè- Ques Académiques: Une Étude Des Utiiisations et Des Perceptions Centrée Sur i’utiiisateur,” n.d., 17; Kate A Pittsley and Sara Memmott, “Improving Independent Student Navigation of Complex Educational Web Sites: An Analysis of Two Navigation Design Changes in LibGuides,” Information Technology and Libraries 31, no. 3 (September 10, 2012): 52, https://doi.org/10.6017/ital.v31i3.1880; Lee and Lowe, “Building Positive Learning Experiences through Pedagogical Research Guide Design.” 

7Heather Brown, Danielle Drummond, and Christian I. J. Minter, “Establishing a Review Process to Evaluate Research Guides,” Medical Reference Services Quarterly 37, no. 4 (October 2, 2018): 367–74, https://doi.org/10.1080/02763869.2018.1514901; Vicky Duncan, Shannon Lucky, and Jaclyn McLean, “Implementing LibGuides 2: An Academic Case Study,” Journal of Electronic Resources Librarianship 27, no. 4 (October 2, 2015): 248–58, https://doi.org/10.1080/1941126X.2015.1092351; Melia Fritch and Joelle E. Pitts, “Adding Bite to the Bark: Using LibGuides2 Migration as Impetus to Introduce Strong Content Standards,” Journal of Electronic Resources Librarianship 28, no. 3 (July 2, 2016): 159–71, https://doi.org/10.1080/1941126X.2016.1200926; Denise FitzGerald Quintel, “LibGuides and Usability: What Our Users Want,” Computer in Libraries 36, no. 1 (February 2016): 4–8. 

8Darcy Del Bosque and Sara E. Morris, “LibGuide Standards: Loose Regulations and Lax Enforcement,” The Reference Librarian 62, no. 1 (January 2, 2021): 9, https://doi.org/10.1080/02763877.2020.1862022. 

9Del Bosque and Morris, “LibGuide Standards,” 11. 

10Del Bosque and Morris, “LibGuide Standards,” 14. 

11Judith Logan and Michelle Spence, “Content Strategy in LibGuides: An Exploratory Study,” The Journal of Academic Librarianship 47, no. 1 (January 2021): 102282, https://doi.org/10.1016/j.acalib.2020.102282. 

12Courtney McDonald and Heidi Burkhardt, “Library-Authored Web Content and the Need for Content Strategy,” Information Technology and Libraries 38, no. 3 (September 15, 2019): 14, https://doi.org/10.6017/ital.v38i3.11015. 

13McDonald and Burkhardt, “Library-Authored Web Content.” 17. 

Competing Interests

The authors have no competing interests to declare.

comments powered by Disqus