Question

How to calculate the course score correctly now that SCORM scores are inserted into the calculation?

  • 26 July 2022
  • 5 replies
  • 409 views

Badge

Now that Docebo has started adding the SCORM values into the calculation of the overall course score, how is everyone handling this? 

Inside each of our courses, we have SCORM and tests placed in different sections (folders), so each section has a series of SCORM files (2004, 3rd Edition) and then a test (created in Docebo). Therefore each course has more than one test. We have the course set to average all scores (which used to be just the test scores, but has recently changed to average ALL scores, including the default zeros that come from SCORM files). Now that the overall course score averages the additional zeros, the score is not correct. 

Summary of setup:

  • multiple SCORM in one course
  • multiple tests in one course
  • each test is worth 100 points, shown as a percentage
  • course > advanced settings > scores are set to “The average of all scores taken by the user in the course.”

Course Example:

  • course has 2 SCORM files and 2 tests
  • each test has 10 questions
  • each question is worth 10 points
  • therefore, each test is worth 100 points
  • user receives a 100 on first test and a 90 on the second test
  • score should be 95: (100 + 90) = 190 divided by 2 = 95
  • instead the user has a course score of 47.5: (100 + 90 +0 +0) = 190 divided by 4 = 47.5
    • the zeros come from the SCORM files, which were not calculated until recently

Has anyone else noticed this issue and what are you doing to get your course scores to display properly? 


5 replies

Userlevel 7
Badge +2

@lrdeardoff I’m not a scorm expert but I think you should be able to turn off the sending of the scorm scores from the authoring tool. What tool are you using? 

I guess the only issue would be if you have multiple evaluations within one scorm. 

Badge

@pmo We use Captivate 2004, 3rd Edition and from everything we’ve read and tested, it’s not possible to turn off the sending of the SCORM. Unless there are any Captivate experts out there who have a different result???

Our SCORM files do not have any scores at all actually, which is why they return zeros. All of our quizzes are created directly inside of Docebo. Therefore, the Docebo quiz scores are being averaged with the zeros from the SCORM files, resulting in incorrect course scores. 

Any help or ideas is greatly appreciated!

Userlevel 2

This  is a problem with multiple layers so you have to sort of peel away certain parts to see the real root of what’s going on. Let’s look at all the layers.

  1. you have 4 objects in a Docebo course/curriculum.
    1. Two objects are content only… no graded assessment, no graded interactions. Created using Captivate and published as SCORM 2004 V3. Completion/success is set for learners to view 100% of slides. With no graded material, the quiz boxes are left unchecked, thus the quiz pass/fail option does not need to be adjusted. 
    2. One object is a content object with an assessment. For this example let’s say 5 questions with each question worth 30 pts. for a total of 150 points. Quiz reporting, completion/success is set to 100% slide views and the “quiz pass” check box is checked. The pass/fail criteria is set to 80% or more of total points to pass. (You need 120/150.)
    3. The last object is a test you built in Docebo. There are 10 questions worth 20 points each. (200 points). You have set minimum score at 80 and set “Score Display “ to express score as a percentage. Max score is set to 200 and you have “Same score for each question” set.
  2. Summary: Four learning objects - 2 do not have  points assigned, 2 have points assigned. Both are scored as a percentage, with both set to 80% to pass.
  3. Under the course properties score and credits tab, you set the course final score calculation to “The average of all scores taken by the user in the course.”

Scenario: Learner goes through all 4 learning objects. They score 80% on the Captivate test and 100% on the Docebo test.  Average should be 90%

Problem: The SCORM packet sends a 0 for Score_raw, Score_min and Score_max, and N/A for the score_scaled entry for both SCORM objects that do not have any assessments. When you publish to SCORM 2004 there is no way to stop any of these  SCORM values from being sent in the packet. (In Captivate, the raw score is the total number of points earned, the minimum score is always 0 and the max score is the sum of all points available. There are NO entry boxes to edit these values to a null and with no questions they would be 0.)

The Captivate object with the assessment sends 120 for score_raw, 150 for Score_max, 0 for Score _min and .8 for score _scaled. (Multiplied by 100 would equal the 80%) (Scaled score never exceeds 1.)

The Docebo object cannot have its SCORM entries reviewed but you can see the 100% on the object review report.

 

Result presented by Docebo dashboard:

0+0+80+100/4= 45 for course/curriculum score

Desired Result:

80+100/2 = 90

Root Problem:

Docebo cannot recognize  a SCORM object within a larger curriculum/course that does not provide assessment data and consequently counts the “0” that comes across all scoring mechanisms by default. I think what @lrdeardoff is asking is:

  1. How many other users have experienced this problem since it is relatively new?
  2. For those that have; have you developed a solution that does not involve hacking your published zip file, “jerry rigging” the SCORM drivers or moving away from SCORM 2004?
  3. What did you do to fix the scores of students who completed the curriculum but have a faulty course/curriculum average?

 

There are several ways you can set up this problem and see the full effect of things but the main underlying symptom in all cases centers around delivering a SCORM object that does not have any scoreable components.

 

 

@lrdeardoff We have also recently encountered this scoring issue in our reports. We also create courses with SCORM 2004 (4th Ed) objects that don’t have scored quizzes, and scored tests built in Docebo.

What we have been able to discover is that since we use Evolve as our authoring tool, there is a ‘Send Score to LMS’ toggle in the SCORM 2004 publish settings that we can disable. We’ve tested that successfully, getting no scores on the SCORMs and the correct average course total scores.

So we have a solution going forward for new courses, but haven’t figured out yet how to deal with our large volume of existing courses, since updating the existing SCORMs might impact our in progress users (yet to be tested). Also, even that wouldn’t resolve our historical reporting for completed users.

Unfortunately, it’s been a few years since I’ve used Captivate so I can’t offer any helpful guidance there. Hoping someone else may have more to offer.

Userlevel 2

@susanf 

Unfortunately Captivate doesn’t have such a button. We started to notice this about 9-12 months ago. Prior to that, things seemed to work fine. What appears to be going on is that, at present, Docebo only works with the raw score sent in the the SCORM packet. I’m sot of guessing that prior to this problem they probably worked with the raw and scaled score. In a captivate SCORM packet the LMS will receive “N/A” as score_scaled and 0.0 as score_raw for those courses lacking any assessment content and published in SCORM 2004. I suspect prior to this problem Docebo had some sort of interrogation of both inputs so that if the scaled score was n/a and the raw score was 0 that it returned a command to not insert any grade for that object into the report. Now that they seem to only work with the raw score that interrogation is gone and the 0 gets put into the report.

I would wonder if the SCORM packet you are sending changes the raw score in the packet to n/a or some non-numeric character. If it did, Docebo probably cannot interrogate it correctly and the 0 is never placed in the report. It’s good you have a solution moving forward, but fixing old scores is still a huge question that I really Docebo would solve. 

I would really question whether or not Docebo is truly SCORM compliant by excluding the scaled score from its operations?? Just my thoughts though.

Reply