In our previous articles we talked about how to choose e-learning courses and distance-learning systems. But we haven’t really looked past that yet. What about the day when all the pros and cons have been weighed, the expenses have been calculated, the courses have been paid for, and a new question has arisen to torment you: how do I know if my courses are actually working or not? How do I evaluate a course’s effectiveness?
Marketing experts have been trying to solve this problem for decades now, developing new analytical methods and tinkering with old ones. In this article we’re going to take a look at traditional metrics and analytical techniques that have already proven themselves and are widely used in corporate environments.
COR (Completion Rate) refers to the percentage of students who finish a given course.
COR can be determined using a simple formula: just divide the number of “finishers” by the number of “takers,” then multiply that number by a hundred. The higher the percentage, the more effective the course is. As a rule, courses taught by an instructor have the highest COR: 70–80%. On average, courses people have to pay for out of pocket are completed by 45% of students. Free distance-learning courses hover around 3–13%.
Low COR usually suggests that students aren’t really that interested in the course. Courses with overly difficult material or excessively superficial information can bore people. But all kinds of things can happen. For example, some people might prefer to study sections that interest them while ignoring the rest of the content. Others might realize they’ve chosen the wrong subject or difficulty level, or maybe the information just isn’t relevant to them anymore for whatever reason. So you always need to assume a small margin of error in your calculations.
NPS (Net Promoter Score) indicates how loyal a course’s students are after it’s done.
In 2003, after studying statistics from over 400 American companies, Fred Reichheld published an article about a new method for analyzing effectiveness. According to his research, the highest loyalty indicators (75%) belonged to giants like eBay and Amazon, whereas the market’s average NPS was about 16%.
But what exactly is NPS, and how is it calculated? The primary criterion is how willing students are to recommend a course they’ve taken to other people. Students’ feelings about the course are evaluated based on a ten-point scale:
Neutral responses are ignored when calculating NPS. Overall loyalty is determined by the ratio of people who are “for” and “against” the course. This data allows us not only to evaluate how satisfied the audience is, but also to predict sales.
In 2021, scholars from the UK and Ireland confirmed that an increase in loyalty has a direct impact on a product’s demand. Their research indicated that increasing NPS by a single point predicts a 1.46% boost in sales during the next quarter. This rule doesn’t apply to longer-term projects, however.
CSI (Customer Satisfaction Index) shows us how satisfied students are with the learning process. At its core, it looks a lot like NPS — students are asked to share their impressions of the course by answering a series of questions:
Students’ satisfaction is determined via a ten- or fifteen-point scale. Follow-up questions help us see big-picture issues such as how certain factors affect the audience’s attitude and which aspects need to be improved. The resulting CSI is the average of all students’ ratings.
By the way, these surveys need to be conducted not just at the finish line, but also at the beginning of the class. This is when it’s easiest to identify the reasons why some students never make it to the finish line.
Intensiveness and assignment completion speed also affect the average impression. If benchmarking takes up the lion’s share of the students’ time, this is a sign you should adjust either the difficulty level or the number of assignments. At the same time, the course should dynamic enough that the students don’t get bored and start nodding off.
Achievement of course objectives is also a very important criterion. This metric evaluates the percentage of students who reach the goals you’ve set for them. For example, if we’re talking about a course on basic computer literacy, students who complete it should be able to not just turn the computer on, but also use a browser and basic software independently.
Focus on the most important metrics. Don’t try to do them all at once. Make testing anonymous so respondents don’t feel embarrassed about expressing their opinions, and your audience will tell you which aspects of your course need improvement.
How to Choose a Learning Management System
Metrics can help up evaluate the effectiveness not only of online courses, but also of corporate training programs. One of the best tools for this is the Kirkpatrick-Phillips model, which has already become a classic. Professor Donald Kirkpatrick came up with it way back in the 1950s, giving us four levels with which to analyze the effectiveness of a course. 40 years later, Jack Phillips added another level. This system gives us five levels of analysis and five key questions that help us get a sense of the quality of a course from various points of view:
Needless to say, the higher the ROI, the better. Although a negative ROI is business as usual for new companies. As a rule, they’re often undermined by a lack of preliminary analysis and the ability to see the big picture.
As Jack Phillips said in an interview, if you invest in people, you need to know what you’re going to get out of it in advance. And the more expensive a training program is, the more important it is to analyze this investment. So you need to reevaluate the results on a regular basis, tracking changes and making adjustments as needed. It goes without saying that you want to make your courses as effective as possible.
If you’re looking for time-tested solutions, remember that we’ve got years of experience, and we’re more than happy to help. We can develop courses for you that factor in your company’s unique characteristics and objectives, and we provide comprehensive support — not just during our initial partnership, but afterward as well. You can learn more about us and flip through our portfolio here.
