Students learn how to access library resources through formal instruction, peer suggestions, and their own trial and error. For years, academic librarians have partnered with faculty to meet students where and when they have a critical need to access the library by attending face-to-face classes and, more recently, by embedding in courses through the LMS. Many library systems and LMSs are integrated to some extent in order to put library resources at students' point-of-need. This literature review focuses on the user attitudes, expectations, and perceptions that impact library-LMS integration decisions.
… While LMS design
appears to not support
emergent trends in student
learning and instructional
design, they are naturally
positioned to do so in the
According to EDUCAUSE, the Learning Management System (LMS) is a ubiquitous tool in higher education with 99% of almost 800 colleges and universities having at least one LMS in place.1 These systems have a sustained presence, with 92% of colleges and universities reporting the use of at least one LMS annually since 2002.2 Additionally, both faculty and students are most satisfied with LMSs for disseminating and accessing course content but least satisfied with collaboration, interaction and engagement functions.3 Blended learning design, a combination of face-to-face and online learning practices, is on the rise in higher education and is projected to continue this growth over the next several years.4 Additionally, over the next three to five years, Johnson predicts deeper learning methods such as, ". . . project-based learning, challenge based learning, inquiry based learning and similar methods . . ." will have great implications for technology in higher education.5 While LMS design appears to not support emergent trends in student learning and instructional design, they are naturally positioned to do so in the future. The confluence of these factors-standard use of LMSs, changes in learning design, and the growing trend of deeper learning—will continue to challenge libraries to develop meaningful integrations with these systems that meet student and faculty expectations in support of student learning.
… Library literature called
for libraries to engage with
LMSs starting in the early
Over the past twenty years, many articles were published about the academic library's presence in Learning Management Systems (LMS). Library literature called for libraries to engage with LMSs starting in the early 2000s. Mick O'Leary, David Cohen, John Shank and Nancy Dewald are often cited as the first wave of researchers highlighting this gap in library services,6 but Donald Beagle found this concept in literature about Asynchronous Learning Networks (ALNs) in the middle to late 1990s.7 At that time, Beagle noted that ALNs were developing to support both web-based learning environments and computer-mediated learning technology. Beagle highlighted a few articles that called on practitioners in this developing field to integrate libraries into ALN work and systems or risk removing libraries from these new learning modes. From his survey of literature, Beagle warned the manual practice of providing course materials in printed course packets would create an expectation that web-based learning environments replicate the traditional manual distribution method in the virtual environment. He also observed a trend in the literature about distance learning practices, which tended to rely on gathered resources for students similar to a course packet, and cautioned this might also result in excluding libraries from online learning tools. As Beagle predicted, LMSs evolved without a default library integration and academic librarians continue to work toward closing this gap.
Shank and Dewald noted concern about the lack of library-LMS integration and wrote about ways to resolve the deficiency. In their review of the possibilities, they acknowledged the lack of consensus about how libraries could blend with learning environments, prompting them to develop models of integration: Macro-Level Library Courseware Involvement (MaLLCI); Micro-Level Library Courseware Involvement (MiLLCI); or a combination of the two. The macro-level integration could be described as a top-level integration of general library resources. A link to the library website homepage or a chat-with-the-library widget, universally visible to students in the LMS, were examples of macro-level integrations. A micro-level integration, on the other hand, was much more customized to a department or even a specific course or section. Examples of this level of integration included links to disciplinary subject guides, embedded customized tutorials, or a librarian embedded in the course. The researchers eventually concluded that a combination of the two integrations was the best scenario to achieve the most comprehensive benefits to students.8
Several articles followed up on Shank and Dewald's macro- and micro-level approaches to library integration. These articles focused on how to overcome technology hurdles when blending libraries with learning management systems. Dan Lawrence provided a primer on making connections with the Blackboard system9, and Dygert and Moeller's use of Blackboard for course reserves led them to implement library resource pages in the LMS.10 In 2008, the perspective in the literature began to shift when two library integration articles were published that included the corresponding evaluation instruments and evaluation results. Washburn surveyed students about their expectations and perceptions of library integration11 and Black reported on students and faculty experiences with library integration.12 While case study articles were helpful for identifying how to integrate library systems with the LMS, more attention on student and faculty needs may help close the gap on meaningful library-LMS integrations in this new era of learning design and deeper learning methods.13
… The authors conducted
an extensive literature
review to determine how
libraries approach user
attitudes, expectations, and perceptions about
library services in the LMS
The authors conducted an extensive literature review to determine how libraries approach user attitudes, expectations, and perceptions about library services in the LMS environment. A number of trends emerged from the research, but only the following four articles reported data about student and faculty attitudes, expectations, and perceptions of library-LMS integrations.
Brigham Young University, Lee Library
When Brigham Young University first adopted Blackboard as their learning management system, there was no integration of library services and resources.14 According to Washburn, e-Learning Librarian at Brigham Young University, that changed after an Annual University Conference student panel requested the integration of library resources into Blackboard, and faculty echoed the sentiment in an evaluation by the Center for Instructional Design (CID).15 Building on the expectation of a library-LMS integration, BYU's Lee Library initiated a blended macro-level and micro-level strategy to incorporate library resources into Blackboard. At the macro-level, general library links were added to the Blackboard system, while automated subject specific links based on each student's class schedule were created at a micro-level. In the article, Washburn shares not only Lee Library's LMS integration implementation strategy and survey results, but also the student survey instrument.
Washburn took a quantitative approach and adapted a survey from Oregon State University Libraries Interactive Course Assignment Project. Students had an opportunity to provide feedback about their experience with Lee Library's integration into Blackboard and whether their expectations had been met. When asked if the page was easy to use, 87% of students responded that it was at least, "OK,"16 which seemed to confirm that the new tool met students' expectations. The survey also indicated that students perceived the tool made a positive impact on their final assignment. Students responded they felt the page, "helped them locate resources for their project" (93%)17 and that their, "research project was better because they used the page" (87%).18 Washburn's 2008 article appears to be the first library integration research to specifically focus on student expectations and perceptions with a research goal to determine whether "students felt that their papers, projects, etc. were improved because they used the resources on the course page."19
Ohio State University Libraries
At The Ohio State University Libraries, a research team integrated library resources into their learning management system by "building a toolkit of systems and options to facilitate creative and flexible interactions between librarians, students and instructional faculty . . ."20 Black discussed implementing links to library resources at a micro-level in LMS course sites. When users clicked the library link, they received a page of library resources based on their status as a student or as a faculty member at either a course level, a department level, or a college level.
Black's research focused on testing the usability of the library-LMS integration, but it included questions about user expectations and perceptions. Like Washburn's 2008 article, Black gathered data from students about their expectations and perceptions of integrating library resources into their LMS course pages; however, Black’s article appears to be the first to gather faculty expectations and perceptions.
Black gathered quantitative use data and a mix of quantitative and qualitative data from both a student survey and a faculty survey, which were shared in the 2008 article. The OSU research team first piloted the Library Resource Pages service in 2007 with 345 courses. Quantitative system use data were gathered but were not reported in Black's 2008 article. Faculty reported strong interest in the service for their students, with 41% stating the desire for course-level pages for each of their courses.21 Students revealed they were more likely to use the library link than faculty.22 Half the student participants shared that they noticed the library link on their course page, but of that group, only half stated they used it.23 Additionally, students perceived the content as "useful" but "a little boring."24 Based on these expectations and perceptions, the group chose to stop creating pages of library resources for faculty and continue exploring how to deliver such materials to students in the learning management system.
University of California at Berkeley Libraries and Colorado State University Libraries
Acknowledging Washburn's innovative evaluation of student attitudes about library integration into the LMS, McLure and Munro initiated a research project to capture student and faculty "attitudes toward the integration of a library presence and library services within course management systems (CMS)" at the University of California at Berkeley and Colorado State University.25 Neither university had a coordinated library presence in their respective learning management systems at the time, so the results of the interviews would potentially determine the initial direction for library integrations. The interviews included sample screenshots of library integrations at other institutions to give the participants a way to visualize how the library could appear in the LMS. The seven samples were a blend of micro- and macro-level integrations.
The authors structured their interviews to collect responses not only about participant demographics, individual experience of the library in the LMS, and ideas for making the library presence in the LMS better, but also preferences for certain features or functionalities integrating the library into the LMS. Interview participants had many opportunities to share their expectations and perceptions, and they did. Participants' perception of the library website was that the system was, "overly complex and hard to navigate."26 While there was an "overwhelming" expectation that library access should be integrated into the LMS consistently across courses,27 several participants commented that library access from the LMS should be easy to find and to use.28
When compared to other library-LMS integration research being done at the time, this research project went in previously unexplored directions. McLure and Munro are the only researchers that chose an entirely qualitative approach to collecting data. Although the results cannot be generalized due to the small sample size, they shared their interview protocols, so that others might adapt and replicate their research process. Another aspect that set McLure's and Munro's research apart was their timing. The authors surveyed their communities before they planned their coordinated library-LMS integration projects. Instead of treating user attitudes, expectations, and perceptions as an iterative part of the library-LMS integration process, they prioritized their users' needs and used the collected responses during their design process.
University of Michigan Libraries
Leeder and Lon address the lack of knowledge about faculty use of library tools in LMSs in their 2014 study.29 The researchers analyze both faculty use and perceptions of a suite of three library-LMS integration tools implemented at the University of Michigan in the late 2000s. The tools were listed along with other LMS components, like Announcements and Assignments, and were set up as opt-in features that faculty could "turn on" for their courses. One feature was a macro-level integration faculty could use to embed the library's instant messaging chat box in their course site. The second was a micro-level integration that automatically connected reserve readings to the course site from another library system. The last was another micro-level integration, a librarian role, which faculty could use to grant a subject librarian access to their course site to interact with students. Unlike Black's 2008 study, Leeder and Lonn focused entirely on documenting both faculty use of the library tools and their attitudes about them.
Referring to quantitative usage data, the researchers verified how often tools in the suite were used. Of all LMS course sites, 17% (773 course sites) operated with Library Tools turned on by faculty over three semesters.30 The researchers used these results to determine the academic departments that most frequently used the LMS Library Tools. Based on this usage data, the researchers surveyed two groups of faculty - the most frequent users of the library tools and those who never used the tools -- in the academic departments with the highest library tool usage. These surveys, included in the appendix of the 2014 article, gathered qualitative and quantitative data covering: demographics, awareness of the tools, interest in the tools, frequency of use, and training. Regarding use, the researchers found that faculty who reported having longer careers and more experience with library resources tended to be more likely to use the suite of LMS Library Tools.31 Most non-users reported they simply did not know about the tools, while many users reported discovering the services on their own in the LMS.32 The authors also identified a contradiction in the data from non-users. Non-users noted they would expect librarians to add resources to a course website and field student questions on that site, but they were also the group who most frequently responded that "using LMS library tools has little connection to my courses."33 The LMS Library Tools were intended to address the faculty expectation of librarian support for their classes, but faculty did not perceive that the LMS Library Tools actually provided that library support. The authors found this issue indicated a gap between expectations and perceptions since faculty did not understand what the tools did based on their associated names and their placement within the LMS.
Leeder and Lonn also compared the perceptions of faculty and librarians regarding the library resources and functions within the LMS. Faculty expected librarians to "answer questions from students online", yet they did not place a high value on the LMS chat service tool. On the other hand, faculty and librarians both placed a high value on student access to library resources.34 The researchers closed their paper discussing how future research into user perceptions of Library/LMS integrations can affect student learning.
Discussion of the Literature
The projects discussed in this paper gathered data from students and faculty to learn about their attitudes, expectations, and perceptions of library-LMS integrations. While their specific goals, methodology, and results varied widely, they all made inferences from the results and suggested iterative changes to improve library-LMS integrations.
… Whether the evaluation
was targeted toward
students or faculty or both,
this research demonstrated
the value librarians place
on how students and
faculty experience the
library in order to improve
One of Leeder's and Lonn's research goals explicitly mentioned perceptions in their research question, "how do faculty's perceptions of library resources and functions in the LMS differ from those of librarians?"35 McLure and Munro also expressed their desire to examine student and faculty attitudes in their research question. Other institutions' research goals implied a review of attitudes, expectations, and perceptions. For instance, Black's research examined how students and faculty felt about the usefulness of the library presence,36 and Washburn's research explored whether students felt the library integration helped them turn in a better assignment.37 Whether the evaluation was targeted toward students or faculty or both, this research demonstrated the value librarians place on how students and faculty experience the library in order to improve student learning.
The researchers highlighted in this review developed strong evaluation plans, but each research project differed in its approach. Even so, the descriptions of the student and faculty evaluations can be used as a blueprint to implement new research projects that focus on user attitudes, expectations, and perceptions. The researchers published their evaluation instruments, which can provide a starting point for designing an assessment plan that accounts for each institution's unique array of resources. Some authors in this literature review did not report their entire research design process, leaving readers to wonder about details like how surveys were disseminated and specific response rates.
Several of the evaluations returned a small number of results. In Washburn38 and McLure and Munro,39 the survey sample sizes were small enough so the results could not be generalized. While the results from the McLure and Munro evaluation were too small to be generalized, their decision to use an entirely qualitative approach highlighted faculty and student expectations and perceptions in a way that quantitative data could not. For example, responses about the feature, "Librarian-moderated discussion forum" were reported as, "mixed, with stronger approval from CSU participants."40 This result, which showed different degrees of approval at each institution, might not have been as obvious if the researchers had chosen a survey or other quantitative research method for their study. Choosing to use the interview method provided the participants an opportunity to share localized expectations and preferences and gave the researchers specific integration targets for each institution, although they did not have objective data to review and, as a result, had much more bias to interpret. For those that relied heavily on quantitative data, they missed an opportunity to review the richness provided by replies to open-ended questions.
Each researcher or research team took a unique approach to their evaluation. In a 2015 article, Farkas concluded, "Librarians should look at how the LMS is used at their institution, their internal, technological resources, ...and the needs of students and individual programs in order to choose a solution that strikes a balance between needs and resources."41 This sentiment applies not only to the technologies libraries employ to connect the LMS and library systems, as Farkas suggests, but also to the unique resources researchers can draw from at each institution to assess student and faculty attitudes, expectations, and perceptions. Institutional culture and availability of resources steered researchers' decisions in everything from the initial wording of research questions to the method of evaluation. Evaluation methods fell on a spectrum from entirely quantitative to entirely qualitative and included mixed methods. As the assessment design process developed, researchers made decisions whether or not to invest in time-consuming research methods like the qualitative approach. Availability of resources also influenced Leeder and Lonn, who collaborated with their human resources department to identify faculty not using the library tools. Not every institution would be able to gather information about which faculty did not use the LMS to survey non-users, but the University of Michigan had these resources. At BYU, the Center for Instructional Design (CID) had a standard for conducting surveys and regularly sent surveys to students about their experiences using the LMS. Part of the institutional culture at BYU was that students were familiar with this pattern of evaluation, and Washburn was able to leverage this familiarity by partnering with the CID to send the library survey. The researchers identified in this article relied on their institutional cultures and available resources when they designed and implemented the assessment instruments and integrated the research results.
The evaluation processes in these articles did not follow a specific pattern, and no one-size-fits-all approach to assessing user attitudes, expectations, and perceptions has been developed. Each researcher or research team created their own process, which generally included making decisions about what audience to target, identifying research questions, selecting research methods to support their goals, and sharing their evaluation instruments. These research projects provide examples for librarians designing projects that explore student and faculty attitudes, expectations, and perceptions to leverage their unique institutional structures and cultures.
Future researchers can begin their assessment planning with ideas from these previous research projects. As researchers design their projects, they have an opportunity to adapt these strategies based on their unique institutional structure. Integrating a library presence into the LMS is a natural evolution of the classic idea of integrating library resources into course content. Regardless of the format, student and faculty attitudes, expectations, and perceptions are the cornerstone of coordinating the library footprint in a given course. We ask ourselves 'how will students conduct research?' and 'do faculty feel library resources contributed to successful student projects?' LMS technology has added another layer of complexity to the puzzle. As technology evolves to address changes in learning design, student and faculty attitudes, expectations, and perceptions will continue to be a critical piece of the course integration puzzle.
Liz Thompson, (firstname.lastname@example.org) Instruction and Educational Resources Coordinator, is an assistant professor at James Madison University whose research interests include student-centered instructional design and open educational resources.
David Vess, the User Experience Librarian & Social Work Librarian, is an assistant professor at James Madison University.
Received: Aug. 15, 2016
Accepted: Dec. 6, 2016
Published: July 31, 2017
© Authors: Thompson and Vess. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source.
1 Eden Dahlstrom, D. Christopher Brooks, and Jacqueline Bichsel, "The Current Ecosystem of Learning Management Systems in Higher Education: Student, Faculty, and IT Perspectives," EDUCAUSE Research Report, (Louisville, CO, 2014): 5, http://www.educause.edu/ecar.
2 Ibid., 8.
3 Ibid., 11.
4 Larry Johnson, Samantha Adams Becker, Michele Cummins, Victoria Estrada, Alex Freeman, and C. Hall, "NMC Horizon Report: 2016 Higher Education Edition," (Austin, TX: The New Media Consortium): 18-19, http://cdn.nmc.org/media/2016-nmc-horizon-report-he-EN.pdf.
5 Ibid., 14.
6 Mick O'Leary, "New Academic Information Model Bypasses Libraries," Online 25, no. 4 (2001): 72-74; David Cohen, "Course-Management Software: Where's the Library?," Educause Review 37, no. 3 (May 2002): 12-13, https://net.educause.edu/ir/library/pdf/erm0239.pdf; John Shank and Nancy Dewald, "Establishing Our Presence in Courseware: Adding Library Services to the Virtual Classroom," Information Technology and Libraries 22, no. 1 (March 2003): 38-43, http://search.proquest.com/openview/e8fd0dc6da68e1f812e5801d17ef0c24/1?pq-origsite=gscholar&cbl=37730.
7 Donald Beagle, "Web-Based Learning Environments: Do Libraries Matter?," College & Research Libraries 61, no. 4 (July 1, 2000): 367-79, https://doi.org/10.5860/crl.61.4.367; Meredith Farkas, "Libraries in the Learning Management System," Tips and Trends: Instructional Technologies Commitee: ACRL Instruction Section, (2015): 1-5, http://acrl.ala.org/IS/wp-content/uploads/2014/05/summer2015.pdf.
8 Shank and Dewald, "Establishing Our Presence in Courseware: Adding Library Services to the Virtual Classroom," Information Technology and Libraries 22, no. 1 (March 2003): 38-43, http://search.proquest.com/openview/e8fd0dc6da68e1f812e5801d17ef0c24/1?pq-origsite=gscholar&cbl=37730.
10 Claire Dygert and Paul Moeller, "Linking the Library and Campus Course Management System," The Serials Librarian: From the Printed Page to the Digital Age 52, no. 3-4, (2007): 305-309, http://dx.doi.org/10.1300/J123v52n03_09.
11 Allyson Washburn, "Finding the Library in Blackboard: An Assessment of Library Integration," Merlot: Journal of Online Learning and Teaching 4, no. 3 (2008): 301-16, http://jolt.merlot.org/vol4no3/washburn_0908.pdf.
12 Elizabeth L. Black, "Toolkit Approach To Integrating Library Resources Into The Learning Management System," The Journal of Academic Librarianship 34, no. 6 (November 2008): 496-501, https://doi.org/10.1016/j.acalib.2008.09.018.
13 Johnson et al., "NMC Horizon Report: 2016 Higher Education Edition." 14-15, 18-19, http://www.nmc.org/publication/nmc-horizon-report-2016-higher-education-edition/.
17 Ibid., 308.
18 Ibid., 309.
19 Ibid., 306.
21 Ibid., 499.
25 Merinda McLure and Karen Munro, "Research for Design: Exploring Student and Instructor Attitudes toward Accessing Library Resources and Services from Course Management Systems (CMS)," Communications in Information Literacy 4, no. 1 (2010): 30, http://www.comminfolit.org/index.php/cil/index.
26 Ibid., 51.
27 Ibid., 53.
28 Ibid., 54.
29 Chris Leeder and Steven Lonn, "Faculty Usage of Library Tools in a Learning Management System," College & Research Libraries 75, no. 5 (2014): 641-63, doi:10.5860/crl.75.5.641, https://doi.org/10.5860/crl.75.5.641.
30 Ibid., 646.
31 Ibid., 648.
33 Ibid., 653.
35 Ibid., 642.
38 Ibid., 307.
39 McLure and Munro, "Research for Design: Exploring Student and Instructor Attitudes toward Accessing Library Resources and Services from Course Management Systems (CMS)," 30. http://www.comminfolit.org/index.php/cil/index
40 Ibid., 38.
41 Farkas, "Libraries in the Learning Management System," 4. http://acrl.ala.org/IS/wp-content/uploads/2014/05/summer2015.pdf