Learning Analytics and courses

Lesson goals and objectives

In this lesson you will learn:

  • how to use Learning Analytics
  • how to use a model to approach Learning Analytics in online courses

Learning Analytics and courses
The main difference between student analytics and learning analytics is the extent to which profiling takes place at individual levels. If you use student analytics on the level of your institution, probably all you need is a data analysis tool like Power BI offered by Microsoft or Google Data offered by Google. And, of course, a skilled data analyst. If you analyse the number of students that complete a course within a certain amount of time, there’s no profiling on an individual level at all. All you know is the number of successful students, no matter who they are.

If you add another variable such as pre-education or age at starting the course there is still no individual profiling. But if you analyse the visiting behaviour in the learning management system of individual students within a course and relate this to their study success, then you are profiling on an individual level. Your intentions are undoubtedly good, because perhaps you would like to support and guide the students online in order to increase their study success or prevent them from dropping out of their studies prematurely.

So the first step in determining an algorithm is to consider which indicators should play a role in assessing the (online) learning behaviour of the student. It is best to start with the actual behaviour indicators, which are easy to measure. These could be the following indicators:

– number of assignments submitted as a percentage of the total in a given period. This is a measure of quantity.
– number of assignments submitted on time – this is before the deadline – as a percentage of the total. This is a measure of behaviour, regularity and dealing with deadlines.
– the scores on the assignments handed in. This is a measure of the quality of the student’s work.
– The number of sources studied. In order to learn, the student must visit and process sources.
– the scores on the knowledge tests. In vocational education, education always consists of a mix of knowledge and skills. A certain amount of theoretical knowledge must be present in the student in order to be able to carry out professional assignments successfully.

What do you want to know?
The first step in applying learning analytics or using student analytics is to describe as accurately as possible what you want to achieve, what you want to measure and for what purpose.

The second step is to use all the information you have in your systems in addition to the data you gather from the learning management system where the real learning takes place. You might want to know what factors within the learning process of the student affect the success of the student in finishing the course. You want prediction from your learning analytics application in order to be able to intervene and adjust at an early stage. This noble goal is meant to make the student as successful as possible, but also to reduce the drop-out grades as far as possible. You seek the best of both worlds.

It is quite easy to identify factors outside the learning process. It is reasonable to suppose that the entry level predicts part of the success. Likewise, the level of unauthorised absence possibly affects the result. But absence of course also can mean other things. It can mean that the student is not motivated to study the course, but also that the content of the course is already mastered, and the student argues that following the course is a waste of time. So, it’s important to investigate what is behind the figures in your data base.

A model to approach learning analytics in online courses
Students who work in an online learning environment leave digital traces. It is important to consider which digital traces should be collected in order to be able to say something (descriptive) or even predictive about the student’s learning behaviour. In most online courses, students must carry out study activities and make work of them. One question could be whether the submission on time, i.e. before the deadline, should be part of the indicators on the basis of which the student is assessed in the model. Most tutors agree that the quality of the work submitted is important to measure. The number of sources studied by the student can also be an indicator of how hard the student is working. And there are often also knowledge tests that give a grade.

In this way, it is possible to write an algorithm that at some point in a course of study describes how students are doing. And that in turn can be the basis for the mentor to have a conversation with the student about adjusting the study path. This adjustment can consist of speeding up or slowing down the study, but also as something to be discussed in a talk about the student’s motivation for the study. In the example below, student 1 needs attention and student 2 is doing well. With student 7 the conversation could be about earlier exams or about following extra modules. With students 3, 5 and 6 a conversation should be held immediately. Their low scores can have many different causes, such as long-term illness, too low a level of prior education, a language problem, lack of motivation or simply a lack of commitment.

As a teacher you will say this looks very familiar to you. In a situation of classroom teaching, the same analyses are done. But keep in mind that we are talking about online education here, where there is much less face-to-face communication with the student and where the behavioural patterns are not visible in the classroom but from the data in the online learning environment.

Note that the number of visits, frequency of visits and length of stay in the learning management system are not included in the algorithm. This is only possible if you are sure that the system records these variables very precisely. Moreover, it is often the case that the processing time of an assignment varies greatly from student to student, depending on prior knowledge of the subject, affinity with the subject and other factors. So, the time spent on an assignment is not always a good indicator of commitment.

However, what might be possible and important to measure is the frequency with which the learning environment is visited. For example, if there is a weekly classroom lesson in which the assignment must be made during the lesson and an online lesson that must be made as homework, you can assume that the student visits the learning environment at least twice a week. In addition to the activity-oriented approach you can decide to bring in digital data on motivation and the ability to reflect on learning. If you want to take the student’s motivation into account and give it a place in an algorithm, you also need to measure it systematically and in a structured way, per learning task, professional assignment or how you have arranged the course.

Since motivation is a strong predictor of study activity, it’s worth trying to include this in the model. You can use the same approach to measure the extent to which the student expresses what he or she has learned from a particular learning task. This too can be systematically and structurally questioned and included in the model. Furthermore, you can also integrate the degree to which the student is able to reflect on his or her own learning behaviour in the model.

In this way, a second view on the total functioning of the student is created, in which motivation, reflection and learning effect are included in addition to the actual activity and scores. You can display this visually in a spider web.

The fellowship of the learning
This is a game to design education and learning analytics. The core of this game is that you cut up all the student activities in steps. You work out all the steps, to each step you can of course link methods and techniques (TPACK), but also learning analytics. What do you want to know per step?

This game is great.  The educational design and learning analytics must be closely linked (really for a Fellowship of Nerds). The concept of first coming up with the interactions needed for learning and then looking at what you want to record to help the student along is very enlightening. You would just like to spend a day frolicking with them, together with other enthusiasts.