The Sources Of Learning Analytics You Should Be Collecting
By now you’ve heard that your Learning Management System is a goldmine of data that can impact your course design, your students, and your company’s bottom line. But if you haven’t used learning analytics before, you might be wondering what data is the right data. What are the best sources of learning analytics in your LMS, and how do you figure out which ones to use?
We often speak of the importance of Learning Analytics (LA) and actionable Learning Analytics in Instructional Design, but we don’t talk much about where to look for them. With the enormous amount of data that’s available, it’s easy to get overwhelmed by the prospect of using learning analytics. But if you start small, you can establish a foundation to build upon as you become more familiar with the learning analytics data that’s available.
The key is to start with the right LA data. In this article, we’ll give you the top sources of learning analytics — and help you think about which ones make sense for you.
Why Are You Collecting Learning Analytics Data?
There are many reasons to collect data for learning analytics. Instructional Designers use LA for a wide range of purposes, including:
- Prediction.
To identify students at risk of dropout or course failure. - Personalization and adaptation.
To give students customized learning pathways, or assessment materials. - Intervention.
Providing instructors with information to support students. - Information visualization.
Learning dashboards that visually display overview learning data.
Before you begin analyzing your data, you should understand what you’re collecting, and why you’re collecting it. Just as your courses have specific objectives and outcomes, your data collection should, as well.
If you try collecting all the data you can find and then try to sort it out later, you’ll only create headaches for yourself - and probably end up paralyzed by the chaos. But a strategic, goal-oriented approach will guide your choices of data sources and help keep your efforts easy to manage.
In general, learning analytics help you understand the effectiveness of your learning modules at 3 levels. You will need to consider which of these levels you’re seeking to understand before you select the data to use:
- Student impact.
Personal performance in relation to learning goals, learning resources, and study habits of other classmates. - Course impact.
This includes data such as social networks, conceptual development, discourse analysis, and “intelligent curriculum”. - Organizational impact.
Predictive modeling and finding patterns of success and failure.
You can use learning analytics to understand the success of your online learning programs at each of these levels. Fleshed out, your desired outcomes might be to answer questions like these:
- What are areas for course improvement?
- How can we spot struggling students early?
- What is the adoption rate of the learning modules?
- Is the course material appropriate for students’ competency levels?
- Are we increasing organizational efficiency?
- How are the online courses helping to reduce costs or increase revenue?
Understand your goals, and map your goals to the type of data available. The data you analyze will be determined by the outcome you want to achieve. And avoid the shiny-object syndrome — don’t collect data you aren’t going to use.
Top 3 Sources Of Learning Analytics
Once you understand why you’re collecting data, you’ll be able to select the most appropriate data sources. Depending on your goals, your top learning analytics will be gathered from one or a combination of the following contexts.
1. Engagement Statistics
Engagement-related statistics can help you assess learner engagement with the module. If you need to measure the success of a learning initiative based on usership, this is a great place to start. Look at these sources for engagement insights:
- Site statistics and logs, location/IP; quiz/course activity statistics (question assessment).
- Logins.
- Course access, time spent “learning”.
- Session – session metrics, tools and content accessed, frequency.
- Enrollment.
- Learner origin of access – home, office, etc
- Device access – mobile or desktop.
- Correlation between course resource access and assessment results.
2. Performance Statistics
These statistics are performance-related. They can help you assess the quality of your courses and the effectiveness of your training modules. You won’t need all of them in your assessments. Only use the data that will be most valuable to achieving your objective — what it is that you want to understand.
- Participation in discussions.
- Participation in the course.
- Gradebook scores – quizzes, exams, homework submissions.
- Self-assessments (graded and non-graded).
- Journals.
- Other course activities – webinars, classroom, collaborative exercises.
- Use of resources (video, pdf, etc.).
- Course progress.
- Leaderboard.
- Frequency of access – how often a resource or activity is accessed and how long.
- Online student feedback—on the course, the instructor, etc.
3. Course Or Site Helpdesk
While this data is external to your Learning Management System, it can be valuable. Helpdesk data tends to be fairly rudimentary, but it can be an important source of information for assessing patterns and frequently-asked questions or issues.
Examples Of Learning Analytics In Action
Imagine that you want to identify individual areas for course improvement. Statistics related to online learner performance, student feedback, and completion rates would help you see the areas where a significant portion of your students struggled. Look for trends that might imply weaknesses in the course design — maybe particular assignments or course components that have a large number of incompletes.
If you want to get a sense of your students’ learning experience in the course, collect data that will tell you what they think about the eLearning activities, the course, and even the instructor. Watch for trends to spot strengths and weaknesses in the typical student experience.
To determine if the training modules are successfully affecting on-the-job performance, use LMS metrics that assess theory-to-practice application. You can also track interactive assessment results from simulations.
Making Learning Analytics Manageable
Learning analytics get a lot of press in the Instructional Design world, but it’s rare to hear much about which LA sources are the best ones to collect. Taking a strategic approach can help make an enormous task manageable. Understand your goals, map your goals to the type of data available, and collect only the data you need.
Start small, and before long you’ll start getting a better understanding of the effectiveness of your online learning programs.
Ready to go deeper in your LMS mastery? Check out our upcoming webinars!
Related Articles:
- Free eBook: Advanced Reporting In Moodle & Totara, With Zoola Analytics
- 3 Tips To Turn Learning Analytics Into Actionable Outcomes
- 7 Ethical Concerns With Learning Analytics
- The Importance Of Learning Analytics In Instructional Design
- Learning Metrics That Matter: Data Points You Should Be Measuring
