Skip to main content

Planned Improvements for Usage Dashboard

  • July 20, 2023
  • 12 replies
  • 1411 views
Planned Improvements for Usage Dashboard
fabio.pirovano
Docebian

Hi everyone,

 

Thank you for your patience as we’ve collected feedback and continued to investigate the issues that many of you have raised regarding the release of the Usage Dashboard.

 

The previous MAU - (Monthly Active Users) tracker was built on legacy code, making it difficult for us to improve and innovate. Understanding how important it is to our customers to have clear insights into their usage of Docebo products, it was critical for us to design a new experience that could scale with improved performance, more easily support future innovation, and allow us to organize billing & contractual information in a single place for our customers.

 

We recognize, now, that we missed the mark in a number of key areas with this release, which we’ll address later in this post. We apologize for any inconvenience the rollout of the new Usage Dashboard may have caused you and your organization. Thank you to each of you who have submitted tickets, provided feedback in the community, and met directly with our account teams, product teams, and leadership to help us understand how we can improve this new feature. We’ve monitored feedback across multiple channels, including the following idea:

 

 

For technical reasons, some of which were shared above, we will not be reverting back to the previous MAU tracker. Rather, we are committed to improving the experience in the new Usage Dashboard to better meet the needs of our community.

 

The purpose of this post is to summarize and centralize the feedback we’ve collected so far, explain our short- and mid-term plans to improve a number of key areas, and provide an opportunity to share specific feedback on a few items.


 

Summary of Feedback

Thanks to many of you in the community, here are the most popular requests to improve the Usage Dashboard that we’ve identified and our plans/timing to address them:

 

  • Short-term = Genaral Availability end of August
  • Mid-term = Updates on timing to be shared before the end of August

 

Request

Being Considered?

Display a table of Active Users within the Usage Dashboard (similar to old table view)

Yes | Mid-term

CSV Improvements

Yes | Short- and Mid-term

Update “De-anonymized” terminology to improve clarity

Yes | Short-term

Add Audit Trail events to track access to Usage Dashboard

Yes | Short-term

Display Usage Dashboard directly in Docebo Learn (reduce total clicks)

Yes | Short-term

Display # of MAUs directly on the counter within Docebo Learn without clicking

No | Technical limitation

 

Continue reading to learn more about how we plan to improve each of these key areas.

 

Display a table of Active Users within the Usage Dashboard | Mid-term

We understand the desire for a table view of active users within a specific time period, similar to the view that was available in the previous MAU tracker. The plan is to deliver a table experience similar to that of the previous MAU tracker, hopefully adding new functionality you’re used to in many other areas of Docebo Learn, like the ability to edit the # of results that display per page, editing which columns display in the table, editing the date range filter, and, of course, downloading a CSV including the filtered results directly from the Usage Dashboard.

 

Given the complexities involved in centralizing billing information, we are actively investigating a number of technical limitations and privacy requirements so that we can work to develop a solution for this request. This enhancement is planned as mid-term.

 

CSV Improvements | Short- and Mid-term

We plan to deliver the following enhancements to the Active User CSV in the short-term:

  • Add a column for Branch Path (similar to the previous Branch Path column)
    • Format: Branch 01 > Branch 02 > Branch 03 (same as previous format)
  • Add a column for Domain
    • Displays Extended Enterprise domain
  • Fix confusion related to User Column by creating three distinct columns:
    • User ID
    • Username
    • Firstname – if empty, column will be blank
    • Lastname – if empty, column will be blank
  • Improve Event Type column to display explicit event that caused Active User trigger for all active users (We will provide the list of possible events in our documentation)
  • Improve clarity for Datetime column
    • This column currently displays the First Access Date. We will update the title accordingly
    • Note: It’s currently not straightforward to include Last Access Date in the Usage Dashboard data. We are exploring two options:
      • Technical enhancement that would allow us to add this information to the Usage Dashboard data
      • Add functionality to allow admins to run a dedicated report within Docebo Learn to gain insights into user login behavior
  • Rework/improve the CSV download workflow to reduce clicks 
    • We are exploring ways to further reduce the number of steps required to download a CSV from the Usage Dashboard. This item will be Mid-term.

 

Update “De-anonymized” terminology to improve clarity | Short-term

We understand that this terminology has been confusing for many of you. In the short term, we will review the naming to clarify the difference. In the mid-term, we plan to remove the “anonymized” version and support a direct download of the “de-anonymized” CSV.

 

Add Audit Trail events to track access to Usage Dashboard | Short-term

Many of you have requested the ability to track which Superadmins within your platform are accessing the Usage Dashboard, and when. We plan to track the following events in the Audit Trail:

  • Superadmin accesses the Usage Dashboard
  • Superadmin downloads the “de-anonymized” active users CSV

 

Display Usage Dashboard directly in Docebo Learn (reduce total clicks) |  Short-term

To improve the experience accessing the Usage Dashboard, we plan to load the Usage Dashboard directly within your Docebo Learn platform, rather than in a new tab or window.

 

Display # of MAUs directly on the counter within Docebo Learn without clicking | Will not deliver

We will continue to allow Superadmins to view the Active Users count in the Usage Dashboard panel upon a single click.

 

Updated MAU Calculations

We’ve received a number of questions about the contractual implications of the new way we calculate MAUs. Docebo's terms have always included ILT activity as part of the calculation for MAUs. The Usage Dashboard now brings this calculation into alignment with existing terms, which is why a contract addendum is not required for this change. This update allows customers to better understand when an active user is being engaged so they can better track usage. 

 


 

We built this community to help you all connect with one another and share your ideas so that in turn we could collect feedback efficiently and respond to it effectively. Thank you for your feedback. We’re committed to collaborating with you to improve the Usage Dashboard to better meet your needs.

 

We’ve prepared some questions to guide us while improving these areas. We welcome your feedback on each of these questions in the comments.

 

Questions

  1. What features and navigation elements are most important to you when considering a “Table view” of active users in the Usage Dashboard?
  2. Would you be interested in a dedicated report in Docebo Learn for user login behavior? If so, what would you like to see in this report?

12 replies

Forum|alt.badge.img
  • Helper I
  • July 20, 2023

This is a great first step! We also need Course Name as a field. The course_id is riddled with blanks when the accessed course in question is Docebo free content, and I’ve been informed that those don’t have IDs to avoid collision with customer created IDs. This leave a major gap in our ability to prove that those users accessed a course at all without massive manual lift. Unless the improved Event Type will cover this? 

I take extreme issue with the demonstrably false claim that docebo’s terms ever specifically included ILT activity in what constitutes an active user. That said, I’d rather focus on improving the product.

  1. No opinion
  2. That would be great! One of the most important series of fields would be 1st MAU triggering access, 2nd MAU triggering access, 3rd, etc. The reason for this is some customers might bill based on the old definition of MAU that was recently replaced without consent. Namely, customers need a way to track the users that would be considered active under the previous definition, as well as under the new one. 

Jessica Tart
Helper II
Forum|alt.badge.img+4
  1. What features and navigation elements are most important to you when considering a “Table view” of active users in the Usage Dashboard? I think this is already new functionality, but being able to separate MAU by domain within the table view would be most helpful. It would also be nice to see some additional metrics, such as MOM and YOY for each domain. I could also see value in being able to filter by ILT vs. eLearning MAU usage as well as active and inactive users.
  2. Would you be interested in a dedicated report in Docebo Learn for user login behavior? If so, what would you like to see in this report? Yes!! We currently have a manual process of downloading the report every day and uploading it into our data warehouse. It would save time for us to be able to setup a scheduled report. We’d like to be able to filter by domain, active users, and course type.
  3.  @kaitlinke is there anything else you’d like to see?

  • Influencer I
  • July 20, 2023

Hi @fabio.pirovano - thanks for your work and response on this. Re-adding Branch path column is especially welcome. 

Before Questions response, please clarify some of the stated improvements:

  • ‘Fix confusion related to User Column by creating three distinct columns’ - seeing 4 bullets here, do you mean Docebo will revert and split the current ‘user_id’ column out into 3 separate fields as before? (username, first name, last name)
  • Datetime column - changing the title is one thing, the data remains an unreadable string, will Docebo be tidying this up to just show date or display date and time legibly? (as with the ‘first access’ , ‘last login’ fields of previous MAU report). This was strongly called for among the Docebo Community. 

    1.  What features and navigation elements are most important to you when considering a “Table view” of active users in the Usage Dashboard?
    # MAUS per sub-Branch , ILT vs eLearning filter, filtering by user additional fields, filtering by content category.
    2.  Would you be interested in a dedicated report in Docebo Learn for user login behavior? If so, what would you like to see in this report?
    First and Last access data points should be standard, clear metrics included within the Usage dashboard csv report. Beyond this, more granular user login behaviour folded into a dedicated report would be welcome, including user branch path, course names accessed, total time spent, time spent per course, user additional fields filter. 

    Moving forward, keen for further info on the ‘mid-term’ updates timeframes. 

     

steveninfinger
Helper III
Forum|alt.badge.img+8

This is good news! Thank you for taking the feedback and taking action. 

  1. Currently, it looks like the only way to get detailed data is by billing dates with Docebo. That isn’t helpful for the data we’re interested for our business. We need more flexibility on the custom range range that we’re pulling, which is usually the first day of the month through the final day, but that doesn’t look like it’s consistent with Docebo’s billing dates. 
  1. I don’t think a dedicated report is necessary, as templated reports rarely have all of the information we’re looking for to answer a question. Have a field available to us to build our own report is more valuable because we can use it to answer different questions based on our needs. First login and most recent log in are examples of helpful info to have in new report. 

 


sabine.deliveroo
Influencer I
Forum|alt.badge.img+1
  1. What features and navigation elements are most important to you when considering a “Table view” of active users in the Usage Dashboard?
    The most important elements to us would be username (e-mail), first access and last access (if possible). Additionally, it would be great to be able to report on all additional fields, doesn’t matter if it’s downloadable from the ‘Usage Dashboard’ or a separate, dedicated active users report.

    2.  With the current way of reporting, it is impossible for learning admins to understand monthly department/subdepartment/level (or other additional field level) active learning per month from a single report. I don’t understand why this doesn’t exist on Docebo. In order to do so, I need to download the active users report and match it (vlookup or other) against a full user report to know the % of active learners per department or any additional field I want to use.

    The advice that I was given was to check number of active users (top of the platform) vs users-courses (last access column) report. So I do this every month. Needless to say, the data never matches - I always have more active users in the dashboard than ‘last access’ users in the users-courses report, therefore our metrics are inaccurate. 

    It would be great that in addition to fixing this dashboard, you introduce a new report, where we’re able to add any of the additional fields we’ve defined in the platform and know exactly what content was learned that month - everything that is a standard to add in other reports.

JeanetteMcVeigh
Hero III
Forum|alt.badge.img+7

thank you for listening to our needs on this one. It was a huge curveball to our processing of information and is taking me a much longer time than it should to get what I need to report out (and well, still is until the data is better). :-)

  1. What features and navigation elements are most important to you when considering a “Table view” of active users in the Usage Dashboard?
    • One click export
    • Would be nice to have the ability to schedule this data as a report to send out regularly 
    • Filter by columns for quicker analysis
    • Be able to choose, on the screen, what data/columns to view
  2. Would you be interested in a dedicated report in Docebo Learn for user login behavior? If so, what would you like to see in this report?
    • Yes, please. Other than doing what another user posted above (running reports daily and adding to a data warehouse (which we do not do)), there is not real understanding of how many times users log in to the system - just the last time. This would be helpful to understand who logs in but does NOT take a course - would help us figure out potentially why they just log in. Potential outreach opportunity to figure out why.
    • As far as what to see, the more data about logins the better. How long per session, how many clicks, what did they click on. If someone doesn’t need that data, they can ignore it. But for us, understanding the users behaviors can help us make things better for the user.

And dare I say it, I am not sure how you missed the mark so much on a release like this. I understand it was legacy code to begin with and needed updating, but an outreach to users before this change might have saved you and us (the users) a lot of frustration. Now, if you did do an outreach, I missed it and I apologize for not jumping on that one.

This usage data is a huge thing for us (and many others as well). I am happy you are responding to our needs...but it could have been avoided (I think).  Always here to help - all of us are from what I can tell. And listen, I get it...if you ask 10 people for an answer, you will get more than you asked for, but you should still ask.  :-)  Kindest regards, Jeanette


Thank you for addressing the concerns raised regarding the recent Usage dashboard changes. 
 
1. What features and navigation elements are most important to you when considering a “Table view” of active users in the Usage Dashboard?
  • First access, Last Access columns need to be re-introduced. 
  • A Number of Login Attempts field would be useful.  Last Access will tell us when they last logged in, but not how many times.
  • De-anonymised CSV file should be available without requesting email access.  
  • There is no useful information in the anonymised CSV report, outside of the total number of active users (which is available on the first page).
  • Change the Chart / Table view, so Table is the default setting. 
  • Alternatively, change the page view, so Chart / Table view displays on this page, with drop-down field views for Learn / Extended Enterprise etc
 
Would you be interested in a dedicated report in Docebo Learn for user login behavior? If so, what would you like to see in this report?
If the above changes are made, a dedicated report would not be required. 
However, the following would be useful:
  • The number of login attempts for each user in a month 
  • Number of active user logins per month per branch

Forum|alt.badge.img
  • Novice III
  • July 27, 2023
  1. What features and navigation elements are most important to you when considering a “Table view” of active users in the Usage Dashboard?

    As outlined above, the current table view functionality we see across Docebo is highly desired.  Displayed rows, editing and sorting columns, filtering based on a variety of formats such as branches, groups, or domains, and downloading this content are all very desirable.  Being able to schedule an e-mail as well, but that may be addressed in question #2.
  1. Would you be interested in a dedicated report in Docebo Learn for user login behavior? If so, what would you like to see in this report?

    We would be very interested in a dedicated report in Docebo Learn looking at User Login behaviour.  In addition to all the standard user information (name, username, branch, etc.) and standard report functionality (scheduling emails, downloading, filtering, etc.) we would love to see:
  • Frequency of login (I think @JeanetteMcVeigh summed this up nicely).
  • Date/Time.
  • Session length, average session length.
  • Actions while logged in (browsing, enrollment, taking a course, completion, etc.).
  • Mode of login (e.g. SSO or manual credentials).
  • Data availability in Custom Reports.

Forum|alt.badge.img
  • Helper I
  • August 29, 2023

When is mid-term? Cannot wait for these fixes!


Forum|alt.badge.img
  • Helper II
  • September 21, 2023

Short-term = Genaral Availability end of August

Mid-term = Updates on timing to be shared before the end of August

 

 

It’s now nearly end of September. Can we have some updates on the timing?


  • Influencer I
  • September 26, 2023

@fabio.pirovano - echoing @Maz above here, awaiting further info as pledged on the mid-term updates - are these coming to fruition? 


dkraige
Helper I
Forum|alt.badge.img+1
  • Helper I
  • February 5, 2024

Any news on this topic?  It seems there has been silence for several months, and it’s still ridiculously cumbersome to get a list of MAUs.  Dozens of clicks, entering email addresses, waiting for emails to come, opening spreadsheet, etc.  Is a solution near?