Checking Under the Hood: Choosing a Learning Management System

Questions For Choosing a Learning Management System

Last month’s post discussed the types of online learning options—two-way video, social networking sites, MOOCs—available to schools, universities and other educational institutions at local, regional and national levels. Options aside, for most educational institutions, a Learning Management System (LMS) is online learning. And because it is often the most expensive component of the online learning ecosystem, because there are dozens of LMSs, and because once you’ve chosen and built on your online program around a certain LMS, breaking up is hard to do (apologies to Neil Sedaka), this post advocates the importance of asking good questions when shopping for an LMS.

[Note: I've gathered these questions from various sources and my own experiences over the years--too many to remember-- so a general thanks to the many skilled e-learning professionals with whom I've interacted, worked and whose resources I've accessed.]

There are many more categories of questions you may want to ask for choosing a Learning Management System, but here are some to start you off:

1. Cost

From experience, buying an LMS is like haggling in a Middle Eastern market, or less exotically, buying a car. There is really no fixed prices and vendors are willing to wheel and deal, especially if your online program will have lots of students (This is great news for national and regional ministries of education. For schools and smaller entities, this speaks to the power of forming buying consortia). Getting the vendor to unpack costs (if they will do that) can help you decide what you do and don’t need and what you will and won’t get. Some questions you might ask are:

• How much does the LMS cost?
• Right. Got it. Now, really, how much will it cost us?
• Are these capital or recurrent costs?
• Are there annual licensing fees or other costs in this price?
• Can you break down the cost into components?
• What is the per user cost? Does this reflect infrequent users and are there special rates for them (for example, someone taking a one-week certification course)?
• What post-implementation costs will we potentially incur?
• How do these costs compare with quotes from other vendors?
• What does a “license” include (e.g., How many people? How many computers?) and for how long?
• Do we have to pay fees when LMS is updated?
• Does the vendor have an enterprise rate?
• What does this total cost include (for example, upgrades, server space, maintenance, learning objects, training, support, etc.?)
• What other costs are involved?
• (This is good place to compare vendor X costs with those of vendor Y and Z and ask more specific questions)

2. Vendor Profile

Given the number of mergers and acquisitions in the e-learning world, you want to make sure that your vendor is stable—that it will be around for a while so you can have consistency with your el-learning system and maintain the relationships you’ve invested in. Possible questions to consider:

• Is the vendor reliable and well-known?
• Does the vendor have documentation (evaluations, testimonials, etc.) that they offer a quality product?
• If you are outside North America, Europe, Australia, does the vendor have a strong presence in your part of the globe?
• Does the vendor have sales, training, and support staff who speak the language of your country?
• Does the LMS have a language pack in your language?
• (Depending on the type of education institution) how much university/secondary school/education market share does the vendor have?
• If the product is open-source, is there a strong development community associated with it and how do we contact them?
• Who are some of the LMS users? (Schools, universities, where?)
• What is the reputation of the vendor and product among other schools/universities?
• If the vendor is marketing an open-source system like Moodle, are they a Moodlerooms provider?
• Does the vendor have research about other ways the LMS has been used? Can the vendor provide references?
• Can the vendor provide list of institutions using this LMS so you can cold call for references?
• What are plans for upgrades? Are there planned mergers/acquisitions/organizational changes? Will the current contract hold in the event of these? How will that impact us?
• Will the vendor provide you with a "divorce clause"? If not satisfied, will they help you migrate your content to another system?
• What is your sense of the vendor staff with whom you are interacting? Do they seem open, responsive, honest, knowledgeable? Do they answer all your questions?

3. Ease of Use

If the LMS is not easy to use--to design for and to learn in/through--people will abandon it in droves and your investment runs the risk of being wasted. And, in fact, some LMSs are quite intuitive and others quite capricious. Some questions to consider:

• How available is documentation, how-to guides, training and online help?
• How responsiveness is technical support (hours, days)? Is it available for instructors and students?
• Will the program require lots of training or is it fairly intuitive to use?
• At a minimum, how long will it take instructors to set up their courses?
• Is the LMS customizable?
• Does it work equally well with all browsers?
• How long does installation take?
• Does it have easy navigation/linking/uploading features?
• Does it come with an FAQ?

4. Access

In particular if you plan on a large-sale system, getting students registered as seamlessly as possible is critical. Some questions to consider:

• Is there individual/group login and password?
• Are there assignable privileges?
• Can instructor manage user profiles, define roles, assign tutors?
• Are any of the features accessible through a browser?
• Can any of this be done via a mobile phone?
• Is there course authorization where instructors approve enrollment?
• Is there registration integration - Registration, Prerequisite Screening, Cancel Notification?
• Can an administrator manage user registrations?
• Can it support Single Sign On provisions with other parts of the LMS or with other programs?
• Can the LMS either build schedules for learners, instructors, and classrooms or synchronize with database managements systems that do?
• Does it offer username and password with forgotten password routine? Is this instant?

5. Course Design

Ease of course design is crucial. Poorly designed courses confuse students and require more of an online instructor (if there is one) in terms of support. Some questions to ask might be:

• Does the LMS allow for easy course structure and design?
• Does the LMS have course templates?
• Does it support all content?
• What types of content does it not support or have problems with?
• Does it support integration of common types of software (e.g., MS Word, Google Docs, etc.) and how well does that work? (For example, if you create something in Word and paste in the LMS, do you then have to go into the HTML editor and clean it up?)
• Can the LMS import and manage content and course ware from other LMSs?
• Does it use and provide access to learning objects? What is the capacity to integrate with existing and newly created learning objects?
• How available is compatible content?
• Does LMS work with most/all open-content and digital content?
• (If needed), how easy is it to author in other languages (e.g., using accents, tildes, cedillas, etc.)
• Does it support web-based authoring?
• Does it support and allow the designer to embed multimedia, video, interactive learning objects/third-party applications (Java-enabled applets, etc.)?
• Is it support accessibility compliance and Universal Design for Learning?
• Does it come with instructional design tools? If so, what are they?
• Does it have pre-integrated content development and/or content management capabilities?
• Does it come with content creation tools? If not, does the vendor have agreements with content creation vendors (e.g., Captivate or Storyline?)
• Can content be created offline and synchronized?
• Is there drag-and-drop functionality? Can multiple files be uploaded at once?
• Does it allow for third-party apps, social media, widgets?

6. Design, Functionality, Integration

The design, functionality and how it works (or doesn't work) with existing systems all impact the user experience. You definitely want to ask the company to create a sandbox where you can test out course design (see above) and collaboration (see below) functions. Beware of vendors that don't allow that. Some questions to consider:

• Is this LMS designed for secondary or tertiary institutions? (If the former, allowing parents to register easily and see their son/daughter's work will be important)
• (If it is a suite), can we unbundle features and purchase components à la carte?
• Is it designed for large or small educational institutions?
• What features does it come with? Can they be modified and turned off/on as needed?
• How do students get registered and authenticated?
• Can LMS support the self-registration process we see in MOOCs? Or is it better done in EMIS or SIS?
• Does the LMS have its own Student Information System? Will it support an existing SIS? If there is no compatibility at the moment, will the vendor create an API so both systems work together?
• How easily does the LMS allow for growth of users, content, functionality?
• Does it/how well does it work on mobile devices? Across mobile platforms?
• Does it adhere to such specifications as QTI, IMS, AICC, SCORM, ISO 9000 or Tin Can/Experience API? Can the vendor provide support for the latter?
• What kinds of analytics does it provide?
• Does it support XML and HTML 5 and is support available for both?
• (This is also a good place to ask compare-and-contrast questions with other LMSs)

7. Online Collaboration and Communications

Communication and collaboration--among students and between students and instructor--are key to a positive online learning experience, so your LMS needs to support such ease of communication. Questions to consider:

• Does it have tools that support community learning or collaboration components that support communication?
• Can it give students and faculty email accounts?
• Can it support e-mail and integrate emails sent from regular POP mail accounts?
• Are there chat rooms?
• Does it have online support /Help Desk? How many hours of each? For students too or only instructors?
• Does the LMS support numerous file exchange types?
• Is there a whiteboard?
• Are there discussion groups/forums?
• Is there a built-in conferencing tool or does it integrate or support the inclusion of third-party tools like Webex or Adobe Connect?
• Do all of the collaboration tools work together well/are they integrated?
• Can students easily access social media tools (like Twitter, Facebook, YouTube, etc.)?
• Is there an online community around this LMS to which instructors and administrators will have access?

8. Assessment

Assessment has traditionally been a weak area in the LMS world, but has improved greatly over the last few years, in part because of the proliferation of school-based online and blended programs. Questions to consider:

• What types of assessments does it support (How easy are these to create?)
• Can the instructor create test questions and facilitate test administration?
• Does the LMS support automated testing and scoring?
• Does it support competency mapping/skill gap analysis?
• Is there a portfolio feature?
• Does it support multiple types of assessment--quizzes, essays, tests, surveys, blogs?
• Are there easy ways to create rubrics and use them seamlessly to assess student work?
• Is there a grade book? Can it be easily exported and imported to  and from Excel and to/from an external grading system?
• Can the LMS print certificates?
• Can the instructor create standard and customized reports on individual and group performance?
• Does it track student participation, progress and performance and inform the instructor when/if there is a potential problem in these areas?

 9. Security

"Security" here assumes multiple levels--you will want network security,  access to the course through secured connections, ensuring that only the student (and instructor and possibly parent) has access to his/her work, secured data and privacy through encryption and third-party certification, and even security in terms of data recovery in the event of a natural disaster or sudden unanticipated major interruption.

• Is the LMS compatible with your nation's (e.g., Children's Internet Protection Act) and/or school's security and privacy protocols?
• Does the LMS handle security or authentication schemas?
• Are there tools for digital rights management (DRM)?
• Are there strong privacy controls?
• Is there encryption (encodes and decodes messages)?
• Is there multiple authentication?
• Will it host the site remotely (if so, where) or support local hosting?
• Where are the servers located?
• Is there an automatic backup system? How frequent?
• Is there a disaster recovery plan?

10. (For open-source systems) Openness

The biggest issue for institutions to consider (I think) is the tradeoff between cost and convenience and support. The old adage in e-learning is that "free" is free like puppies, not beer. There will always be costs, even in a free system, and these costs may manifest themselves in greater amounts of support staff time. Questions to consider?

• How open is the source code? Is it available and if so, from where?
• What are the languages it’s written in?
• If it is open, is it also free?
• Is it written in a modular format that is designed for easy modification and new, customized modules?
• Are there clear code specifications for writing new modules?
• Does the code come with supports/documentation?
• How active and involved is the open community in supporting this LMS?
• What's more important to our organization--cost of convenience and support?

Conclusion

The above list of questions --there are certainly more that could be asked--demonstrate that the decision to employ a certain LMS should be made carefully and after a great deal of research and great deliberation (Unfortunately, this is often not the case). The LMS is not your e-learning program, but it is the vehicle that transports your students to a certain destination--and not all e-learning systems are the same. Some are of far better quality and far more appropriate for your learners than others. Taking the time up front  to check carefully under the hood and ask the right questions will serve educational institutions, their instructors and learners well over the long haul.