Best Answer

Standard practices for tracking completion for users taking courses

generic name
Rank
Userlevel 4
Badge

Hi everyone. I have only been using Docebo for a few months now and we are trying to work around some nuances Docebo has such as making sure all our users are enabling cookies. Even with that we are having a hard time pinpointing why people are taking courses but then not showing complete. they will get to the end take a screenshot or picture and send it to us showing us its complete but then logging back into Docebo it still shows as in progress and usually only half way through the course.

 

Any help or pointers on if our users are doing something wrong to cause this to happen?

icon

Best answer by Annarose.Peterson 19 May 2021, 17:59

View original

22 replies

A
Userlevel 2
Badge

Hey there! We’ve had a similar issue with courses not registering as complete (though I didn’t realize it might be tied back to the cookie settings..). We got around it by requiring courses to have a quiz at the end, and then tracking based on quiz completion instead of page views. There was also a more manual process of updating the SCORM file in notepad which would trigger completion, but this was clunky and clearly not scalable. Hope this helps! 

generic name
Rank
Userlevel 4
Badge

Hey there! We’ve had a similar issue with courses not registering as complete (though I didn’t realize it might be tied back to the cookie settings..). We got around it by requiring courses to have a quiz at the end, and then tracking based on quiz completion instead of page views. There was also a more manual process of updating the SCORM file in notepad which would trigger completion, but this was clunky and clearly not scalable. Hope this helps! 

thank you @adriancross, we also have tests in 90% of our courseware, unfortunately even with the tests this issue still seems to be occurring. we added a survey at the end as well to help mitigate this but even that still hasn't reduced it much. Sometimes we get surveys from the completion of the course and still doesn't show complete hahaha.

S
Rank
Userlevel 6
Badge +2

Hi, we have had the same issue, but making sure all of our user use Google Chrome has solved a lot of our issues as the platform works the best on this browser. Most of our users even power users who use a different browser where having issues with even finding the Admin Gear, etc. We are trying to make our courses even better in the course management > course shell > training material - by having 2 separate training resources, the video and then the quiz using Docebo quiz builder in the central repository. This has worked out great for us. But using Google Chrome has resolved a lot of issues as well.

I hope this helps!

generic name
Rank
Userlevel 4
Badge

Hi, we have had the same issue, but making sure all of our user use Google Chrome has solved a lot of our issues as the platform works the best on this browser. Most of our users even power users who use a different browser where having issues with even finding the Admin Gear, etc. We are trying to make our courses even better in the course management > course shell > training material - by having 2 separate training resources, the video and then the quiz using Docebo quiz builder in the central repository. This has worked out great for us. But using Google Chrome has resolved a lot of issues as well.

I hope this helps!

Thank you @Swatson, in the past we also had our users rely on Chrome as well but with Docebo we though we could get away from that and allow users the use of Safari also. its also of note that 99% of our courseware is scorm based. Typically we build our scorm files and then add a quiz at the end as well as a link to a survey within the scorm. Once they click the link that sends completion of the course AND opens the survey we require them to complete.

abartunek
Rank
Userlevel 7
Badge +2

Hi @generic name, I imagine the issue is that some training materials inside of a course are not being registered as completed? When you go to the Course → Reports → Click on a username, it can be easily checked which Training Material is at fault and stops users from completing the whole course.

What I could recommend is adding a Test or a “thank you for taking part in the course” HTML Page at the end of the course AND also marking it as an End Object Marker. Any Training Materials marked as End Object Markers make it so that once they are finished, the status of the whole course changes to Completed (no need to complete any other materials then, unless you set Prerequisites).

You can mark a training material as an End Object Marker by clicking on the Hamburger Menu next to the training material → Settings → End Object Marker → Yes.

The other Adam B.
generic name
Rank
Userlevel 4
Badge

Hi @generic name, I imagine the issue is that some training materials inside of a course are not being registered as completed? When you go to the Course → Reports → Click on a username, it can be easily checked which Training Material is at fault and stops users from completing the whole course.

What I could recommend is adding a Test or a “thank you for taking part in the course” HTML Page at the end of the course AND also marking it as an End Object Marker. Any Training Materials marked as End Object Markers make it so that once they are finished, the status of the whole course changes to Completed (no need to complete any other materials then, unless you set Prerequisites).

You can mark a training material as an End Object Marker by clicking on the Hamburger Menu next to the training material → Settings → End Object Marker → Yes.

Thank you @abartunek, i have not tried that approach yet! i will give that a shot immediately! right now all of our course shells only have 1 training material within the shell, and we have had this issue on multiple courses. Hopefully that will solve this issue.

Adam Ballhaussen
Rank
Userlevel 7
Badge +3

@generic name there are some excellent tips in this thread. Unfortunately SCORMs can often be finicky, buggy, and unpredictable. They are fantastic when they work well, but wildly frustrating when they don’t work as intended. Here are my recommendations based on feedback others have already shared:

  1. Ensure your users are using Google Chrome – Chrome makes everyone’s lives easier and as @Swatson mentioned is our recommended browser.
  2. Use Tests at the end of your courses – tests have a number of great benefits. They help your users retain the knowledge gained in the course, help you measure the effectiveness of your training content, force learners to pay attention, and can act as a reliable end object marker.
  3. Use a Test or non-SCORM learning object as an End Object Marker​​​@abartunek’s recommendation is a great one. I really like the idea of having a single learning object serve as the “key” to being marked complete for a course. You can still track % progress of the course as a whole, but using a single LO to mark completion simplifies the process and makes it easier for you to troubleshoot.

There are some helpful resources in the Training Material Management section of Docebo Help that might be worth referring to. Here are a few that might help you:

While not directly relevant to this topic, you might find the following thread interesting. A few members shared ideas on ways to avoid “fast-clickers” and many of the solutions relate to tracking completion and testing knowledge.

 

Adam B.
Annarose.Peterson
Rank
Userlevel 7
Badge +5

Hi @generic name 

We were having the same problem, even though we tested all of our courses and ensured people were able to complete the SCORM learning objects.

Interestingly, we found that the biggest culprit for courses not completing is because their session timed out. We have session lifetime of 30 minutes.  If a user walks away from their computer in the middle of a course and stays inactive for 30 minutes, the session times out, but the course continues to play.  So when the user comes back and completes the course, the platform doesn’t recognize the completion because they are technically logged out.

Not to say this may be the cause in your platform, but this puzzle left us scratching our heads for a while, until we discovered the time out was our culprit!  

 

Annarose Peterson (She/Her)
joanna.lay
Rank
Userlevel 5
Badge +1

Hi @generic name 

We were having the same problem, even though we tested all of our courses and ensured people were able to complete the SCORM learning objects.

Interestingly, we found that the biggest culprit for courses not completing is because their session timed out. We have session lifetime of 30 minutes.  If a user walks away from their computer in the middle of a course and stays inactive for 30 minutes, the session times out, but the course continues to play.  So when the user comes back and completes the course, the platform doesn’t recognize the completion because they are technically logged out.

Not to say this may be the cause in your platform, but this puzzle left us scratching our heads for a while, until we discovered the time out was our culprit!  

 

We’ve had this as well and it was the session time out that was the issue for us too.  I’d recommend either shortening your SCORM courses (and perhaps splitting them into short modules) or if possible getting the SCORM to report back at a key stage in the course.

We also found the issue was improved by switching to SCORM 2004 for some courses.

Joanna
generic name
Rank
Userlevel 4
Badge

@Annarose.Peterson  @joanna.lay

Interesting! we just had our first person advise us on that exact issue. we have seen some people spend 12 hours in a course that is only 2 hours long. so this would make a lot of sense for some people.

 

Thank you!

Gary Jarvis
Rank
Userlevel 5
Badge +1

I am not sure forcing users to use Google Chrome should be a valid answer to this problem, Docebo states that is fully compatible with the latest releases of multiple browsers. I have over 200 major customers around the world who mandate the use of Edge and have strict SOE rules about adding additional browsers (security)  may be Docebo should look deeper into this issue.

best regards,

Gary

dklinger
Rank
Userlevel 7
Badge +6

Just catching up and reading above? I can see that I landed up summarizing more that was covered in the thread….but I hope this can still help out folk.

Hi - Let me add to this - where people may think that SCORM may feel wildly frustrating while trying to debug them? For a long time user of SCORM? There are tremendous benefits to them…including allowing you to go way outside of the box when it comes to working within the frame of eLearning and lets not forget their portability. 

If you buy a course from a vendor? Chances are they still publishing out to SCORM or a “skinny” version of it.

In our industry? Other standards have been slow to percolate up that are widely adopted. AICC, xAPI, and LTI are neat, but getting outputs from authoring systems may drive you to want to get a developers advice.

That said? Some otherbest practices with publishing to SCORM?

  1. Publish to SCORM Cloud to check and debug your courses. This may seem like an extra step? But the folks at SCORM Cloud really get into trying to answer the question of “whats wrong with my course?” if you need to ask them. And you can setup a free account and trial a few courses without a problem. (Sorry about the product pitch, but Rustici Software? Those folks are seriously out there to assist the community when it comes to courses - some of their folk participate in writing the communication standards to learning systems).
  2. SCORM 2004 vs SCORM 1.2 is more verbose by its nature. SCORM 2004 use of sending along CMI interactions to an LMS? Makes their scoring nearly human readable.
  3. Be really sure about the options the learning system supports when it comes to “listening  for a completion”. Most authoring softwares offer a variety of sending along a completion status (COMPLETE/INCOMPLETE) and/or success status (PASSED/FAILED). But not all LMS are ready to listen to both.
  4. Find a recipe and stick to it. Even though these are seriously aging protocols and I pay attention to the xAPI and CMI5 game? Usually a recipe of the output options will make you successful.
  5. Try to figure out if the LMS went “out of session”. Many LMS’s will essentially freeze their session clock to allow for long SCOs to be taken. But what happens when a course does not submit information consistently? Well the LMS can think it is quiet….and essentially drop a person from their session due to lack of interactivity. A great thing that can be turned on in a few softwares? Is to send the LMS interaction data…think of that as “pinging” the LMS with the interactions from the Course.
  6. Docebo supports best fits for most stability with SCOs and their course player.  Start at most stable and work outwards.
  7. It was said above - but let me reiterate this - TEST TEST TEST. Nothing worse than creating a great looking Rise course to figure out that the completion criteria is not doing the right thing? Or that bookmarking is acting up. Or both.

 

Lucy.blake
Rank
Userlevel 5
Badge +2

We had an issue with it not showing as complete in Docebo. This was when we use Articulate 360 (Rise or Storyline) to create a Scorm file.

Make sure when you export the Scorm file you mark reporting as “passed/incomplete” Docebo doesn’t recognize “Complete/incomplete” as a completion.

dklinger
Rank
Userlevel 7
Badge +6

Hi @Lucy.blake - I think that may be more of a Rise issue. Rise is a great product for responsive design and has the bells and whistles folk need to move content into the future. But I also have found issues with Rise courses with bookmarking and scoring that drive me to pick on it and say - “reserve the tool for basic stuff”. Someone is gonna think I am crazy for saying this - Articulate products continues to be the tool of choice for many next to the Adobe Suite of eLearning tools - with Storyline 360 acting as my go to hammer in the business.

@Adam Ballhaussen - is there a way to verify Lucy’s assertions other than on our own?

 

A
Rank

Jumping in here as the session timeout was also an issue for us recently (and we are only just about to launch our LMS). I’d like to recommend a feature enhancement that allows for an optional pop-up notice that alerts users that they’re about to be logged out and an auto-logout based on the session lifetime. I’d even take it one step further to allow session lifetime to be set at the user level to address the issue with users looking like they’re spending inordinate amount of time in courses. For example, a “user” could have a session logout of 5-10 minutes, but power users could be set at 60 minutes (or more).

 

Would be curious to know how many power users/super admins would support these changes? 

Adam Ballhaussen
Rank
Userlevel 7
Badge +3

 

 

@Adam Ballhaussen - is there a way to verify Lucy’s assertions other than on our own?

 

 

@Lucy.blake @dklinger I’ve certainly experienced my fair share of “head scratches” due to SCORM reporting inconsistencies between Rise and Docebo, but I’ll say they’ve improved substantially over the past year. Earlier this year, our team ran a few tests with various completion reporting options from Rise in our Docebo Sandbox and found that every reporting option (Passed/Incomplete, Passed/Failed, Complete/Incomplete, Complete/Failed) functioned consistently as expected. See the screenshots of the CMI reporting from our four test Training Materials below:

 

Passed/Incomplete reporting option selected in Rise
Passed/Failed reporting option selected in Rise
Complete/Incomplete reporting option selected in Rise
Complete/Failed reporting option selected in Rise

 

@pmo and @ryan.woods could you share the exact export settings we use for our Rise courses in DU?

 

As @dklinger mentioned, it’s always important to test thoroughly to make sure things are working as expected!

Adam B.
L
Rank
Userlevel 5
Badge +2

@Adam Ballhaussen  In your test results it looked like all settings reported a completion.

Did you have any report as ‘complete’ when it should have been incomplete?

Was your course completion based on quiz results, or pages viewed?

C
Rank
Userlevel 4
Badge +2

We have been facing a similar issue since we started using Docebo. This has happened with the scorm packages we developed and also with the course purchased from the Docebo content marketplace. Few users will send us a screenshot of the course completed congratulation message and say it is not registered with Docebo. When we look at the report, their status will be In progress and the SCORM report will look like this.

And our users confirmed that they completed the course in a single sitting. So I don't think, session time is the reason for this.

L
Rank
Userlevel 5
Badge +2

@Chelsy Koshy  Do you know if your learners were clicking the internal (meaning it was part of the SCORM package, not Docebo’s) close button on the course? 

This KB article indicates that is important.

https://help.docebo.com/hc/en-us/articles/360020126739-Marking-Training-Material-as-Complete

C
Rank
Userlevel 4
Badge +2

@lhubbard They were clicking on the exit button of the SCORM package, not Docebo’s.

gstager
Rank
Userlevel 7
Badge +5

We were having the same problem, even though we tested all of our courses and ensured people were able to complete the SCORM learning objects.

Interestingly, we found that the biggest culprit for courses not completing is because their session timed out. We have session lifetime of 30 minutes.  If a user walks away from their computer in the middle of a course and stays inactive for 30 minutes, the session times out, but the course continues to play.  So when the user comes back and completes the course, the platform doesn’t recognize the completion because they are technically logged out.

Not to say this may be the cause in your platform, but this puzzle left us scratching our heads for a while, until we discovered the time out was our culprit!  

 

Hmmm… good tip.

Filed away under “Just in case I experience this...”

Greg
Adam Ballhaussen
Rank
Userlevel 7
Badge +3

@Adam Ballhaussen  In your test results it looked like all settings reported a completion.

Did you have any report as ‘complete’ when it should have been incomplete?

Was your course completion based on quiz results, or pages viewed?

 

Hi @lhubbard! We didn’t have any instances where the training material reported as Complete when it should’ve been Incomplete. The completion was based on quiz results in Rise.

Adam B.

Reply


Terms & ConditionsCookie settings