VALib v61n1 - Just How Open? Evaluating the "Openness of Course Materials in Massive Open Online Courses"


Massive Open Online Courses (MOOCs) are poised to alter dramatically the current and future higher education landscapes. Through various providers, ranging from university-sponsored consortia such as edX, through for-profit partnerships like Coursera, to independent ventures such as Udacity, collegelevel education has been made available to any person in the world, provided they have Internet access. 1 This differs quite drastically from the tuition-based enrollment model of the past and present, which is not only predictable but is also the standard on which higher education financial management is based. Though MOOCs have existed in some form since 2008, 2 their proliferation into mainstream media occurred in 2012, as Laura Pappano declared by titling her The New York Times feature article “The Year of the MOOC.” At the time of Pappano’s article, over two million students worldwide were registered for courses offered by edX, Coursera, and Udacity. 3

Some of these challenges
have begun to be
addressed within academic

The participation of universities in MOOCs continues to grow at a rapid pace. In July 2013, Coursera had 68 university and museum partners, and in June 2014, 99 university, museum, and institute partners, making it, by far, the largest MOOC platform. 4 Similarly, the edX consortium, initially a partnership between the Massachusetts Institute of Technology and Harvard University, has added the University of California, Berkeley and the University of Texas system as members since its founding. While there may be a variety of reasons for universities to join a MOOC system, including reaching potential future students from all corners of the globe, or educational altruism, the adoption rate of MOOCs by the world’s leading universities is high and may continue to increase.

As more universities enter this still burgeoning field, the libraries that support these institutions are challenged with myriad issues, ranging from copyright and fair use through potential reference support of MOOC students, to the more minute details of liaison work with faculty partners. Some of these challenges have begun to be addressed within academic libraries. In October 2012, the Association of Research Libraries (ARL) published an Issue Brief by author Brandon Butler that focused on five legal and policy-related issues regarding MOOCs and research libraries: use of copyrighted works in instructional materials, the assignment of copyrighted works for outside reading, copyright status of materials created by faculty for their MOOC courses, if notice-and-takedown provisions of the Digital Millennium Copyright Act apply to MOOCs, and disability and accessibility. 5 OCLC Research and the University of Pennsylvania hosted a two-day conference in March 2013, “MOOCs and Libraries: Massive Opportunity or Overwhelming Challenge,” which brought together leading authorities on copyright, open access, e-learning, information literacy, and learning technologies to discuss the wide range of issues that MOOCs raise for libraries, including those addressed in the ARL Issue Brief. 6 In March 2014, Carmen Kazakoff-Lane published a white paper with the Association of College & Research Libraries (ACRL) that undertakes an environmental scan of MOOCs, Open Educational Resources (OERs) and the potential impact for libraries in the open education movement. 7

MOOCs are not the first “open” movement in higher education. For over a decade OERs have been explored as a potential solution to the ever-rising costs of textbooks. 8 University libraries are also stakeholders in this discussion as students turn to the library for current editions of their course textbooks either on reserve or in their circulating collections. In 2011, the University of Massachusetts Amherst (U Mass Amherst) Libraries partnered with the Office of the Provost at their institution to develop the Open Education Initiative (OEI) to form a “grant-incentive program to change or augment the traditional textbook model with resources that are openly available or available to students at no additional charge.” 9 Faculty members from a variety of departments participated in a combined 26 courses; using course enrollment figures, Billings et al estimate that over 1,600 students could be impacted by the OEI at U Mass Amherst each time one of these courses are taught in the future. This example of OERs in use for traditionally enrolled students at a university serves as a potential model for MOOCs where other options, including course reserves, are not an option.

… are the reading materials
included by professors
freely available as well?

This article will examine the “openness” of the reading materials listed in MOOCs offered by Coursera and edX, the two major universitysponsored MOOC platforms. Since the major draw for MOOCs is their free enrollment for anyone, anywhere, are the reading materials included by professors freely available as well? If so, what kinds of freely available reading materials are being included? Are OER textbooks and other forms open access content listed by course professors?

Literature Review

Though MOOCs are a new factor in library and information science, as the dearth of peer-reviewed, scholarly articles in the field suggests, education researchers have been examining MOOCs for several years. This literature review will examine a selected number of relevant articles that address pertinent issues in MOOCs that relate to this study, and will not be exhaustive. Liyanagunawardena, Adams, & Williams provide a comprehensive review of the scholarly literature about MOOCs, categorizing the literature alongside quantitative analysis and identifying gaps in the literature; this is a recommended resource for a background on MOOC literature, particularly the educational theory behind MOOCs. 10

Fini focuses on the technological aspects of MOOCs, in particular an examination of user data supplied from a survey at the completion of the MOOC Connectivism and Connective Knowledge (CCK08), and discovered that most of the participants in CCK08 were adult, informal learners, who were not concerned about course completion. 11 Similarly, deWaard et al, found that out of the 556 people who participated at some point during a course titled “MobiMOOC,” only 74 were active members, and of those, 32 were “memorably” active. 12 Attrition in MOOCs is a running theme both in the scholarly literature and in popular resources. Hu notes that attrition rates should be more closely examined in her article for the trade publication Diverse Issues in Higher Education . 13

Morgan and Carey examine the opportunities inherent in MOOCs, arguing that MOOCs can reach those in geographic and/or economic isolation, and can aid in improving academic literacy in English. This article, written before the rapid MOOC expansion of 2012, proposes that traditional universities can offer benefits to the institution and the open education movement by offering open courses with an online delivery as seen by the work of the OpenCourseWare Consortium, and concludes that open course models might be particularly beneficial in disciplines that have a global perspective for a global audience. 14 Pence takes a different approach, examining whether or not MOOCs can provide a stable and successful business model for partner institutions, noting that universities may want to use MOOCs to collect data and learning analytics to better understand why students succeed or fail, and could implement the technological capabilities available in MOOC platforms in course management software, to track time in class, among other factors. 15


This study entailed a close analysis of the course page and syllabus listed for each Coursera and edX course that began between January 1 and March 31, 2013. To gain access to these materials, the author registered for each class offered by Coursera and edX during this time period. The author took screen shots of each course page and syllabus, examining each for any mention of the words “text,” “textbook,” or “readings.” Whenever one of these words appeared on the course page or syllabus, the screen shot was marked to highlight these words. The author then categorized the resources as either “textbooks” or “readings.” The principle distinction here was how the professor introduced the material, and what the item actually was. For instance, when an introductory textbook appeared under “Course Readings,” it was categorized as a textbook. Readings, primarily, were articles from a variety of sources, including web publications, web sites, online newspapers, blog posts, and a few scholarly articles, among others.

Each occurrence of a course material, be it a textbook or reading, was then judged against a matrix of “openness.” For a textbook, a work was considered open access if it was linked to freely, via an OER platform or as a PDF with a Creative Commons license. Textbooks that were not linked to freely, but were linked to through their publisher site or another online retailer, such as Amazon or Google Books, were categorized as “pay texts.” The matrix for course readings was the same. Course readings could be deemed “freely available” or “pay.” While these are broad categories, they generally captured the content, from examples such as National Institutes of Health articles (free) to lists of citations to scholarly articles (pay). One unaccounted for factor not included in this study is the prevalence of readings that are currently freely available, but will likely be behind a pay wall should the course site be archived and be accessible in perpetuity. Included in this category of readings would be online newspaper or magazine articles, which now are freely available but, depending on the publisher and their archiving policies, may be behind a pay wall at some point in the future. For this study, readings that were freely available at the time of the course are included in the “freely available” category.

… the author registered for
a combined 95 courses
offered through Coursera
(80) and edX (15).

There is a risk for error in this data collection and categorization as the author served as the sole arbiter of what was considered a textbook or readings based on the definitions above. Additionally, the matrix devised to judge the levels of open access was also solely the work of the author, and therefore may contain bias in categorization.


From January 1 through March 31, 2013, the author registered for a combined 95 courses offered through Coursera (80) and edX (15). These courses spanned a wide range of disciplines, from earth sciences to economics, and nutrition to Ancient Greek heroes. In all, there were 61 classes in the sciences, 23 in the social sciences, and 11 in the humanities. The largest sub-discipline was computer science, which itself had sub-categories available through Coursera; there were 21 computer science courses offered during the period of analysis. This finding is unsurprising, given that several of the early MOOCs were often computer science-related. 16


In total, 49 courses listed a textbook on the course page or syllabus, referring to it as a textbook, or listing it among other readings. Of these 49 courses, 20 were freely available, or open access; 29 were considered pay texts, with either a link to a publisher site, Amazon or Google Books, or just a citation listed with no link. Coursera courses comprised 13 of the courses that included an open access textbook, which accounts for 16 percent of their total offerings, while 7 edX courses had a freely available textbook, which is 46 percent of edX’s total offerings. It should be noted that the edX platform, which is fairly consistent in the categories and link locations from course to course, allowed for these freely available textbooks to be embedded within the course site. This made accessing the texts very easy, and did not require the students to leave the course site. Conversely, the freely available textbooks in Coursera were listed in different places from course to course, and depending on the platform of delivery for the open access textbook, widely varied.

If Coursera’s user experience in accessing the freely available textbooks was less than optimal, the variety of providers for the textbooks was noteworthy. Several different OER providers were included in Coursera courses, including Connexions, Bookboon, WikiBooks, and the Open Learning Initiative. Additionally, one course had an independent OER: a textbook, Mooculus, written and developed exclusively for the MOOC course “Calculus One,” offered by The Ohio State University. In 4 of the courses, links were included to PDFs of a textbook or chapters from a textbook. One course had 3 chapters available in a PDF file, with no noticeable Creative Commons license; permission to include these chapters, which were written by the course instructor, may have been obtained from the publisher, Cambridge University Press. Another course textbook, also authored by course instructors, was available for free download as a PDF, or available for sale via iTunes, the publisher, or Amazon. In edX, express permission was granted for use of some of the textbooks embedded in the course site, but was not indicated for all 7 courses with freely available textbooks.

The “pay textbooks” include texts that were available via links to publisher sites, book retailers, or through no link at all, simply listed as a citation; these encompassed nearly 60 percent of the courses that listed a textbook. Out of the 29 total courses that included pay textbooks, 25 of them were from Coursera, and 4 were from edX (with one course taught twice during the time period of analysis). A total of 9 courses included links to Amazon, 7 to the publisher site, 2 to, 1 to CourseSmart, 1 to Google Books, and 1 to a personal web site. Of the 9 courses that included links to Amazon, 4 additional options to iTunes, nook, or Kindle editions. The remaining 8 courses listed a citation for the textbook, but provided no link to purchase the book from a retailer. The inclusion of these “pay textbooks” provides a barrier for the students in MOOCs. Even though a number of the course pages and syllabi included language such as “this course is self-contained,” and “refer to the textbook for a deeper understanding of the course content,” students may feel they are not getting a fulsome experience without these textbooks, and may end up unnecessarily purchasing them.

In many ways, MOOCs are
the Wild West of higher

Course Readings

In total, 44 combined courses contained readings or links to readings that were not textbooks; 38 from Coursera and 6 from edX. Of the total, 29 of the courses had readings available from a link or page header with the word “Readings” appearing somewhere (variations include Suggested Readings, Recommended Readings, and Reading Assignments). The remaining 15 courses used a variety of names for where the course readings could be found, with 8 naming them Resources. There were 31 courses with course readings that had links to or embedded only freely available resources. These freely available resources include news and magazine articles, published items from United States federal agencies, web sites, blog posts, and other textual sources, and could be linked to as web resources or available as PDFs for download. Not included in the 31 courses with only freely available resources are the 6 courses that contained both freely available and “pay” readings (in the form of citations with no link or directions on how to access them). These courses listed a variety of readings side-by-side, or in one instance, listed in separate sections.

A finding of note was the use of scholarly articles made available through an institutional repository. In the course “Women and the Civil Rights Movement” offered by the University of Maryland College Park, the course professor included a variety of resources under the heading “Readings.” While the readings did include “pay” readings in the form of citations with no link under the “Supplemental” section, several scholarly articles authored by the course instructor were linked to the University of Maryland institutional repository, DRUM (Digital Repository at the University of Maryland). This use of scholarly content via an institutional repository was the only instance noted by the author in this study. As Paul notes in his study on benefits and incentives and the institutional repository, faculty members are still reluctant to deposit materials in their institution’s repository. 17 A possible strategy from this solitary example of using deposited scholarly content would be for liaison librarians to speak to their faculty about depositing their work in the institutional repository to make their work available for the masses. There is no mandate for faculty authors to deposit scholarly works in DRUM, yet the course instructor did so and was able to leverage her scholarly output for educational purposes in her MOOC. This could be particularly useful if faculty authors have a noted piece of scholarship they would like to make available for colleagues anywhere to link to without worry of copyright infringement.

Of the remaining courses, 7 of them contained only “pay” readings in a list of citations, or links to commercial sites such as Amazon, to purchase a copy of a resource. These appeared in 3 occurrences under the headings “Suggested Readings,” or “Additional Readings.” As previously noted, these headings also appear above lists of freely available resources, which may cause confusion for the student, in addition to an expectation to seek out the content or purchase it.

Textbooks and Course Readings

In total, there were 8 courses that included both a textbook and course readings; 6 of these were Coursera courses, 2 were from edX. While 6 of the total 8 courses had textbooks that were “pay texts,” 2 courses had freely available textbooks. There were 3 courses that had a “pay text,” and all freely available course readings and 3 courses that had a “pay text” and “pay” readings. The 2 remaining courses had both a freely available textbook and freely available course readings; these were both edX courses.

No Textbooks or Readings

There were 10 total courses that contained neither a textbook nor course readings. All of these were Coursera courses, and included 6 in the sciences, and 2 each in both humanities and the social sciences. These courses were truly self-contained, as they relied on no content but the video lectures, assignments, and discussion boards.


In many ways, MOOCs are the Wild West of higher education. Though their adoption has been rapid, with nearly all of the most prominent research universities in the United States, and over dozens more from around the world participating in either Coursera or edX, much remains to be studied and learned about these courses. As stated in the literature review, there are not yet any scholarly studies in the field of library and information science that have become part of the literature. There are, however, editorials, thought pieces, and literature reviews that have been published in the last two years. 18 In her editorial column, Pritchard notes that it is rare to have such an impactful transition occurring so rapidly in higher education, and encourages librarian and information professionals to view it as an opportunity for research and should “…start now to design studies that will give us solid findings demonstrating the successes or failures of contrasting approaches to providing curricular support and research instruction [in MOOCs].” 19

This study is one such examination. By analyzing what course materials are being included in the still early days of MOOCs, librarians can learn where best to start conversations with their faculty who currently teach or may teach a MOOC in the future. For MOOC instructors, the findings of this study may aid in course planning and the selection of course materials. The inclusion by professors of OERs and freely available course readings point to an embrace of the “open” potential for MOOCs. How might this impact standard in person or online classes? If liaison librarians also embrace this spirit in their conversations with their faculty constituents, perhaps open access materials will be the majority of course materials in a follow-up or future study, particularly if faculty members continue to deposit their scholarly work in their institutional repositories.

There are many opportunities for future research on this topic. A comparative study over time would be of interest to see if the adoption of open content increases as libraries become more invested or involved in the hosting of institutional repository content, or in the creation of OERs. Additionally, how do changes with content publishers may or may not make available impact what professors include in their courses? 20 And, taking the long view, how might student interaction with open content in MOOCs drive their expectations for course materials in for-credit, face-to-face or virtual classes? As MOOCs continue to be developed and adapt to their evolving environment, these questions and myriad others will lay poised to be examined in greater detail.

Gene R. Springs joined The Ohio State University Libraries in January 2014 as the Business Librarian. His current research interests include: open access, collections analysis, and succession planning.


1 Larry Johnson, Samantha Adams Becker, Michele Cummins, Victoria Estrada, Alex Freeman, and Holly Ludgate, “NMC Horizon Report: 2013 Higher Education Edition,” Austin, TX: The New Media Consortium. Retrieved from: .

2 Ibid.

3 Laura Pappano, “The Year of the MOOC,” The New York Times (New York, NY). November 2, 2012.

4 “Partners,” Coursera , accessed May 27, 2014, https://www. .

5 Brandon Butler, “Massive Open Online Courses: Legal and Policy Issues for Research Libraries,” Association of Research Libraries Issue Brief , last modified October 22, 2012, .

6 “MOOCs and Libraries: Massive Opportunity or Overwhelming Challenge,” OCLC Research , accessed May 27, 2014, .

7 Carmen Kazakoff-Lane, “Environmental Scan and Assessment of OERs, MOOCs, and Libraries: What Effectiveness and Sustainability Means for Libraries’ Impact on Open Education,” Association of College & Research Libraries , accessed May 27, 2014, org.acrl/files/content/publications/whitepapers/Environmental% 20Scan%20and%20Assessment.pdf .

8 David Wiley, Cable Green, and Louis Soares, “Dramatically Bringing Down the Cost of Education with OER,” Center for American Progress , February 7, 2012, http://www.americanprogress. org/issues/labor/news/2012/02/07/11167/ dramatically-bringing-down-the-cost-of-education-withoer/ .

9 Marilyn S. Billings, Sarah C. Hutton, Jay Shafer, Charles M. Schweik, and Matt Sheridan, “Open Educational Resources as Learning Materials: Prospects and Strategies for University Libraries,” Research Libraries Issues: A Quarterly Report from ARL, CNI, and SPARC , September 1, 2012, .

10 Tharindu Rekha Liyanagunawardena, Andrew Alexander Adams, and Shirley Ann Williams, “MOOCs: A Systematic Study of the Published Literature 2008–2012,” International Review of Research in Open & Distance Learning 14, no. 3, accessed May 27, 2014, index.php/irrodl/article/view/1455/2602 .

11 Antonio Fini, “The Technological Dimension of a Massive Open Online Course: The Case of the CCK08 Course Tools,” International Review of Research in Open and Distance Learning 10, no 5, accessed May 27, 2014, http:// .

12 Inge deWaard, Sean Abajian, Michael Sean Gallagher, Rebecca Hogue, Nilgun Keskin, Apostolos Koutropoulos, and Osvaldo C. Rodriguez, “Using mLearning and MOOCs to Understand Chaos, Emergence, and Complexity in Education,” International Review of Research in Open and Distance Learning 12, no 7, accessed May 27, 2014 .

13 Helen Hu, “MOOC Migration,” Diverse: Issues in Higher Education 30, no 4, accessed May 27, 2014 http://diverseeducation. com/article/52317/ .

14 Tannis Morgan and Stephen Carey, “From Open Content to Open Course Models: Increasing Access and Enabling Global Participation in Higher Education,” International Review of Research in Open and Distance Learning 10, no 5, accessed May 27, 2014, irrodl .

15 Harry E. Pence, “When Will College Truly Leave the Building: If MOOCs are the Answer, What is the Question?” Journal of Educational Technology Systems 41, no 1, 29.

16 Liyanagunawardena et al. “MOOCs: A Systematic Study of the Published Literature 2008–2012,” 215.

17 Shampa Paul, “Institutional Repositories: Benefits and Incentives,” The International Information and Library Review 44, no 4, 200.

18 Bernd Becker, “Connecting MOOCs and Library Services,” Journal of Behavioral & Social Sciences Librarian 32, 135–138; Paul Bond, “MOOCs and Serials,” Serials Review 39, 258–260; Bruce E. Massis, “MOOCs and the Library,” New Library World 114, no 5/6, 267–270; Kerry Wu, “Academic Libraries in the Age of MOOCs,” Reference Services Review 41, no 3, 576–587.

19 Sarah M. Pritchard, “MOOCs: An Opportunity for Innovation and Research,” portal: Libraries and the Academy 13, no 2, 127.

20 Kevin Carey, “Into the Future with MOOCs,” The Chronicle of Higher Education , September 3, 2012, accessed May 27, 2014, With-MOOCs/134080/ . VL