Behavioral indicators

Modified on Wed, 17 Aug, 2022 at 10:34 AM

Summary

In StudyCoach it is possible to monitor data regarding the (online) behavior of students. This data of learning analytics is retrieved from Canvas and clustered into a score per behavioral indicator per week. The data is then collected per course enrollment, per course, and per student via a trend per indicator.

Data is retrieved from Canvas into StudyCoach on the following four behavioral indicators:

Attendance
Page views
Participation
Submission behavior

It is possible to turn on or off the data-collection per behavioral indicator in StudyCoach, this can be done by a manager in the settings.

Value

Insights from the various behavioral indicators can be used to provide an indication of a student's engagement and activity in their courses.

Roles

Student	Teacher	Coach	Department Coach	Manager
✓	✓	✓	✓	✓

Note: these features can be individually enabled or disabled by the manager.

Table of contents

➤ Data collection

➤ Trends per behavioral indicator

➤ Trend calculation per behavioral indicator

StudyCoach offers the possibility to monitor (online) behavior of students by a set of learning analytics. The data collection takes place through Canvas and is translated by StudyCoach into a number of indicators that are scored and then a trend analysis is applied to them. Each step is explained below.

Representation of the process of data collection, translation to scores, and calculation of trends.

Data collection & Scores

Canvas collects various behavioral data from students. Of these, StudyCoach distinguishes and unlocks four types.

Attendance
Degree of attendance based on attendance records.
Page views
Ratio of page views to the average.
Participation
Ratio of active participations to the average.
Submission
Extent to which student turned in assignments on time.

For each of the four indicators, a weekly score is calculated that can be viewed in StudyCoach by course in the form of circles with signal colors. By hovering over a circle with the mouse, the underlying data is displayed in a tooltip.

Score indicators

For more information on the data collected, the calculation of the scores, and the meaning of the signal colors, see the corresponding articles.

✨ Tip: Are you looking for ways to improve the Canvas environment to ensure complete and accurate data in StudyCoach? Or are you a teacher and looking for inspiration? Then dive into this article.

In addition to these four behavioral indicators, the last log in in Canvas and the result per subject are also shown in StudyCoach:

Last login indicates when a student last signed in to the Canvas environment.
Grade by course shows the current grade for each course in the StudyCoach dashboard. For visualization of these scores, a grading scheme can be selected.

Trends per behavioral indicator

For each behavioral indicator, a trend is calculated based on the scores of the past period. Depending on the setting by the manager, the emphasis is placed on the short, medium, or long term. Trends are represented by different circles and a signal color. By hovering over a circle with the mouse, the underlying data is shown in a tooltip.

Trend indicators

The trends can be found in various places in StudyCoach and are intended to provide an initial signal, after which a coach can decide to dive deeper into the underlying data.

Trend per student (for coaches, teachers, department coacher and managers)
This trend provides a summary of the trends across courses and can be found in the Students tab.
Trend per course (for department coaches and managers)
This trend provides an average of the trends of all students and can be found in the Courses tab.
Trend for a single course (coaches, teachers, department coaches and managers)
This trend averages the trends of all students and can be found within a specific course.

Trend calculation per behavioral indicator

The trend per behavioral indicator is calculated via the exponential smoothing method.

Formula with 0 < α < 1

The larger the α, the more the trend per indicator takes into account the last week and thus the short term;
The smaller the α, the more the trend per indicator takes into account the longer term.

α = 75% focuses on the short term and is best used when StudyCoach is consulted weekly and the coach wants to monitor mainly based on the data from the last week.
α = 50% focuses on the medium term and is best used when StudyCoach is deployed 1x per 2/3 weeks with emphasis on monitoring the last few weeks.
α = 25% focuses on the long term and is best used when StudyCoach is consulted a few times per period/semester such as during evaluation interviews, etc.

The variable α for trend calculation and percentages per color can be adjusted by the manager at the root account level in StudyCoach via Settings > Calculation.

✨ Tip: The settings of the trend calculation are important for the representativeness of the Learning Analytics for the purpose of the institute. One of our previous clients, Thomas More, works in the following way:

During the semester, coaches focus on the short term α. Coaches monitor each week to see if any data has changed.
At the end of the semester, the manager changes α to long term. Then the coaches need data that gives an accurate picture of the entire semester, rather than a weekly view.

If the calculation of α changes during the semester, this should be clearly communicated to all users. We would like to recommend that the reason behind this change be specifically explained to all users.

Questions? Feedback? Need help?

Feel free to contact us.