Best Practices for System Validation

Katie,

As an initial platform validation?

We ran a series of test scripts that were driven by role permissions and a “base user experience” in the system. We actually had a vendor go through establishing the steps to document and validate that the environment is acting as expected.

We documented each behavior that was expected, known concerns about the permission set, and then findings by running the script itself to verify that components were working as expected or not expected.

As a backup, we kept the test users that were used in the system so we dont lose the audit and have the document on record with us.

Hope this helps.

Hey there - actually sales and marketing started with Docebo Learn and we are looking to expand the use to our quality system training. Right now I am reviewing the SOC 2 report for key points of design control, change management, data backup and recovery and e-signature. For updates for Cloud-systems where changes are pushed, I have a decision tree that will need to be revised as it is written for quarterly patching not as frequent as Docebo monthly routine. In sum, we will bounce off of the SOC audits, reviewing them annually and review monthly updates for new functions that impact quality system tasks. From there, decide level of risk and testing requirements. Certainly open to other ideas to make monthly reviews less resource consuming while still having a validated system from FDA/Notified Body perspective. And then of course, the API being developed to hook into our document management PLM system...that’s another validation to be determined.

Hi KatieB

Classification of updates as (1) bugfix, (2) modified existing function, (3) new function or (4) deleted function (with ID#).

For new functions a link to a test summary abstract indicating
-  the testing performed by Docebo and
-  if there were any other functions impacted by the code change of the new function along with assurance there was no negative impact to the existing code (or if there was an impact what was it).

To clarify the latter point, a piece of existing code “A” gets altered in order to make new code “B” work. Does the code “A” change affect any other existing code? It’s the potential cascading impact to also be concerned about. 

Hello Katie - I have asked our internal Training Director (Jamie) to reach out to our Customer Success Rep regarding this string as it is important to us. Hopefully we can see some motion forward on some this before we go live with our Quality Management System training deployment. Thank you for raising this topic.

Hi Katie,

My quality team has asked if Docebo can provide evidence of Part 11 e-signature compliance? As we need to validate the system, it would help tremendously to be able to that documentation. I will ask our account manager as well.

Also, they have asked several times if Docebo has a validation package, at a minimum a set of test scripts that can be used for initial system validation, up to a service where this work could be completed on our behalf.  - I have asked this question before and the answer has always been no.

As noted above, accurate and detailed release notes are also needed.

Great question, @KatieBonilla!

With out recent implementation, we had Docebo migrate our content from our previous vendor. The best we could do at validating that everything made it in was to compare the number of rows in the CSV with the number presented in the bottom left corner of Course Management when filtered for a specific course type. Beyond that it required a lot of manually checking each course.

It would have been great to have reporting capabilities in New Reports that allowed us to get more data around our courses. In fact, this is still greatly needed post implementation. The existing reports are all user centric, but we also want to know about the courses themselves. We want be able to run a report on all courses to see data like title, length, for sale or not, and pretty much all of our additional fields like more detailed course type information, the product the course relates to, the curriculum level, and more. It would also be helpful to see the assigned instructor of a session. We’ve had to add an additional field at the session level and add and maintain our 60+ instructor roster so we can see who the instructor for a session is within the course management dashboard and not have to do multiple clicks within the course and sessions. The downside to our solution is additional fields can’t be sorted on the course management page, so it’s only partially helpful when we want to look at a specific instructor’s sessions clustered together.  Being able to report on course data would be such a huge win in terms of validation. When we make system-wide changes, it would be great to make the changes whether manually or by import, then export a report to make sure everything was updated properly. It would also be helpful when it comes to importing sessions and events that require system ID that can only be retrieved by looking in the URL or through API.