In 2023, LTS evaluated and tested three Learning Management Systems (LMS): Canvas, Brightspace, and Moodle. Based on the final overall rankings using standardized procedures and rubrics in three testing environments, Moodle (“Course Site”) was selected as the LMS that best aligns with our needs. This document summarizes findings from the Lehigh University Learning Management System (LMS) evaluation process, encompassing user testing, feature analysis, and feedback.

Additional Factors:

Evaluated LMS Platforms:

• Canvas by Instructure

• Brightspace by D2L

• Moodle (“Course Site”)

Evaluation Methodology:

• Rubric-Based Scoring: A detailed rubric was developed to assess each LMS across key areas, including course migration, design and layout, content organization, communication, assessment tools, and more. Evaluators assigned scores based on their hands-on experience.

• User Feedback: Evaluators documented specific observations, highlights, and concerns encountered during their testing of each platform.

• Feature Checklists: Comprehensive feature checklists were used to confirm the presence and functionality of specific tools and capabilities within each LMS.

Key Findings:

• Overall Ranking: Based on the combined rubric scores across all evaluators, Moodle consistently achieved the highest overall ranking, followed closely by Brightspace, with Canvas receiving the lowest score.

Moodle Strengths:

• Robust Feature Set: Moodle offers a comprehensive set of features and functionalities out-of-the-box, excelling in areas like assessment tools, gradebook, and student tracking.

• Customization Options: Moodle is highly customizable, allowing for extensive tailoring of the user interface, course structure, and functionalities to meet specific pedagogical needs.

• Open-Source Nature: As an open-source platform, Moodle benefits from a large and active community, contributing to its ongoing development and offering a wide range of plugins and extensions.

• Excellent Migration Tools: Excellent migration tools with great documentation. All migrated course materials are ready to use.

Brightspace Strengths:

• Intuitive User Interface: Brightspace is praised for its modern and user-friendly interface, potentially requiring less training for faculty adoption.

• Built-in Features: Brightspace boasts several integrated features, including student portfolios, awards/badging systems, and robust media recording tools.

• Administrative Tools: Brightspace offers powerful administrative tools and reporting features, facilitating efficient course management and data analysis.

• Strong Content Authoring: Provides a suite of tools for authoring media-rich content, importing content, drag-and-drop interfaces, as well as uploading rich content types such as podcasts, video clips, etc. Allows metadata creation for easier/better management.

• Customizable Homepage: “Navigation is at the top as opposed to the side... Can adjust font size... Widget-based platform...Can customize landing experiences...”

Canvas Strengths:

• Streamlined Design: Canvas features a clean and simplified design, potentially appealing to users seeking a more straightforward user experience.

• Ease of Use for Basic Tasks: Canvas excels in the simplicity of basic tasks, such as creating assignments, quizzes, and managing grades.

• Integration with External Tools: Canvas seamlessly integrates with various third-party tools and services, such as Google Apps and Turnitin.

• Group Management: Canvas offers intuitive group management features, including self-selection, importing, and group sets.

• SpeedGrader: "Speed grader is easy to use -still an overlay – but no disappearing comment bubbles, comment library built in, works more like a google doc."

Areas of Concern:

Canvas:

• Limited Functionality in Native Tools: Several essential features require reliance on external tools or workarounds, impacting user experience and potentially increasing administrative burden. No direct poll/feedback tool.

• Lack of Granular Tracking: Student tracking features may not be as in-depth as other platforms, limiting insights into student engagement and progress.

Brightspace:

• Gradebook Usability: Some evaluators found the Brightspace gradebook less intuitive and user-friendly compared to other platforms.

• Potential Extra Clicks for Navigation: The interface may require additional clicks to access core course content compared to other platforms.

Moodle:

• Potentially Steeper Learning Curve: The extensive customization options and vast feature set can lead to a steeper learning curve for new users, requiring dedicated training and support.

• Interface Design: While Moodle 4.1 introduced a more modern look and feel, some evaluators still perceived its interface as less visually appealing compared to Canvas or Brightspace.

Categorization of Funtionality and Features

Course Migration

1. How do the Learning Management Systems (LMS) under consideration handle course migration?

• Canvas: Offers robust course import and export capabilities using industry standards like IMS Content Packaging. It allows copying, archiving, and transferring files and assessments between courses.

• Brightspace: Provides migration tools for moving content from other systems, including Moodle, though some manual adjustments might be necessary post-migration.

• Moodle: Excels in course migration, offering excellent tools and documentation. Migrated content generally requires minimal formatting adjustments.

Design and Layout

2. What are the key design and layout considerations for each LMS?

• Canvas: Features a user-friendly interface, prioritizes simplicity, and supports responsive design for various devices. It offers templates and blueprints for course consistency and has an offline mode.

• Brightspace: Employs a clean, widget-based interface that allows for customization, branding, and personalized landing experiences. It prioritizes visual organization and a consistent look across courses.

• Moodle: Offers extensive customization and personalization options, responsive design, and the ability to work offline. It scores high on design flexibility but can feel more complex to navigate than Canvas.

Content Authoring

3. How do the LMS platforms support content authoring and multimedia integration?

• Canvas: Provides seamless integration with Google Docs and multimedia uploads within assignments. It streamlines content creation and has a user-friendly "SpeedGrader" for feedback.

• Brightspace: Features built-in media tools, making it easy to upload and manage audio and video content. It supports content templates for consistency and offers screen recording and video note options.

• Moodle: Offers robust content authoring tools, including support for H5P interactive content and SCORM files. It excels in flexibility but might require more technical expertise than the other options.

Assessment and Feedback

4. What are the strengths and weaknesses of each LMS in terms of assessments and feedback?

• Canvas: Offers a straightforward assessment tool with various question types, rubric integration, and the ability to provide immediate feedback. It emphasizes simplicity and ease of use for instructors.

• Brightspace: Features a comprehensive assessment system with diverse question types, learning objective alignment, and support for multimedia within quizzes. It excels in its integrated feedback tools and the option for anonymous grading.

• Moodle: Provides a powerful assessment engine with advanced question formats, adaptive testing capabilities, and robust feedback options. It offers the most flexibility but can have a steeper learning curve for instructors.

Communication and Collaboration

5. How effectively do the LMS options facilitate communication and collaboration?

• Canvas: Facilitates communication through announcements, discussions, and in-LMS messaging. Its group features are intuitive and support various grouping methods.

• Brightspace: Offers communication features such as announcements, discussions, and a built-in student portfolio system. It emphasizes collaborative learning with peer review tools that incorporate multimedia.

• Moodle: Provides a comprehensive set of communication and collaboration tools, including forums, chat, wikis, and workshops. It excels in flexibility but might require more setup and management from instructors.

Gradebook and Tracking

6. How do the gradebooks and student tracking features compare across the three LMS systems?

• Canvas: Presents a simplified and user-friendly gradebook that focuses on essential functionality. Its tracking tools are clear but less detailed than Moodle's.

• Brightspace: Includes a functional gradebook, though its color scheme and documentation could be improved. It offers analytics dashboards and tools to identify students needing support.

• Moodle: Provides the most feature-rich gradebook, allowing for extensive customization and detailed student tracking. It excels in tracking student activity and progress, including time on task.

Administrative Features

7. What are the essential administrative tools and integration capabilities of each LMS?

• Canvas: Offers robust administrative features, including course templates, reporting tools, and integration with third-party applications via LTI. It emphasizes ease of use and efficient course management.

• Brightspace: Includes customizable dashboards, a data hub for comprehensive reporting, and robust administrative tools for managing users, courses, and integrations. It excels in providing administrators with granular control.

• Moodle: Boasts extensive administrative options, allowing for fine-grained control over system settings, user roles, and plugins. Its strength lies in its flexibility, but it can be more complex to administer than other systems.

Support and Resources

8. What types of support and resources are available for each LMS platform?

• Canvas: Provides extensive online documentation, webinars, training courses, and an active community forum. It is known for its comprehensive and accessible support materials.

• Brightspace: Offers online documentation, training opportunities, and a dedicated client success team for institutional support. It is committed to providing personalized assistance and guidance.

• Moodle: Has abundant documentation, community forums, and an online academy offering courses and certifications. It benefits from a large, active user community and extensive open-source resources.

Criteria for LMS Evaluation

The sources provide comprehensive checklists of criteria used to evaluate the three LMSs: Canvas, Brightspace, and Moodle. These criteria were organized into categories: Within each of these categories, the checklists included specific functionalities and features that were evaluated for each LMS.

Examples of Evaluation Criteria:

• Page Design: Evaluators considered elements like the ability to drag and drop files, create different types of pages and lessons, include additional information panels, embed videos and rich content, customize configurations, resize for responsive devices, import content, and integrate calendar tools.

• Text Editor: This category focused on text options like heading styles, fonts, colors, link options, tables, accessibility checking, importing images and multimedia, and creating lists.

• Assignments: Evaluators assessed assignment features such as file upload options, rubric integration, feedback types, grading types, access restrictions, polling tools, group submission capabilities, due date settings, and integration with external tools like Google Assignments and Turnitin.

• Quizzes: Quiz functionalities were reviewed, including various question types, the ability to upload annotated work, feedback mechanisms, timed quizzes, access restrictions, question shuffling, question banks, activity completion triggers, multiple attempts, partial answer options, media integration, rubrics, and integrity tools.

• Communication: This section focused on communication tools within the LMS, such as announcements, Q&A formats, discussion boards, group access restrictions, forum grading, availability and due dates, student ratings, subscription options, emails, and notifications.

• Gradebook: The evaluation examined gradebook features like manual assignment entry, different gradebook types, assignment grouping, weighted categories, extra credit handling, export/import functionality, and student retention/warning systems.

• Admin Settings: This broad category covered administrative aspects such as training and documentation availability, performance dashboards, course import/export options, course resetting, offline use capabilities, course templates, standard themes, external participant access, textbook publisher support, and available plug-in and LTI integrations.

In addition to the detailed checklists, evaluators also used a rubric to assess each LMS based on their experience. The rubric included criteria such as:

• Course Migration

• Design and Layout

• Content Organization

• File Management & Exchange

• Content Authoring

• Communication

• Calendar and Adaptive/Selective Release

• Collaboration (Social Learning Tools)

• Group and Learning Community Features

• Gradebook and Tracking

• Testing and Assessment Tools

• Reporting Tools

• Archives

• Training Materials

• Textbook Publisher Support

• 3rd Party Integrations

• Online Help Resources

• Data Syncing/Banner

Each criterion within the rubric was scored on a scale of 0 to 2, with 2 representing exceeding expectations, 1 meeting expectations, and 0 failing to meet expectations.