Add more usage data to EE ping
This MR loads the EE usage ping asynchronously on the admin application settings page and now includes the counts of the following items:
- Comments
- Groups
- Users
- Projects
- Issues
- Labels
- CI builds
- Snippets
- Milestones
- Todos
- Pushes
- Merge requests
- Environments
- Triggers
- Deploy keys
- Pages
- Project Services
- Issue Boards
- CI Runners
- Deployments
- Geo Nodes
- LDAP Groups
- LDAP Keys
- LDAP Users
- LFS objects
- Protected branches
- Releases
- Remote mirrors
- Web hooks
Closes #997 (closed)
Merge request reports
Activity
@stanhu can you indicate in the body of this MR if you add any other data to collect than what is described in https://gitlab.com/gitlab-org/gitlab-ee/issues/997 ?
-
@yorickpeterse @jschatz1 Given that these counts might take some time to calculate, should we require the user to click on a link/button to show the payload?
-
Perhaps we can also load this asynchronously.
Edited by Stan Hu Added 1 commit:
- 9520711d - Add push count
Added 1 commit:
- 9520711d - Add push count
Mentioned in issue #997 (closed)
@stanhu Loading it asynchronously seems like the least annoying solution for the user.
Added 1 commit:
- da6e77b2 - Load usage ping payload asynchronously
Added 1 commit:
- 55ce239b - Add more usage data to EE ping. Load it asynchronously to
Milestone changed to %8.12
Reassigned to @rspeicher
Reassigned to @yorickpeterse
-
I'm assigning this to @yorickpeterse because I think @rspeicher is traveling Monday, and I want to make sure this gets in 8.12.
Reassigned to @stanhu
Mentioned in issue #808 (closed)
Added 112 commits:
-
55ce239b...3e684d89 - 110 commits from branch
master - 98538f9b - Add more usage data to EE ping. Load it asynchronously to
- 71160e90 - Don't use inline JavaScript to modify application settings
-
55ce239b...3e684d89 - 110 commits from branch
Added 1 commit:
- 317a28fd - Move usage data URL into data-endpoint
Added 1 commit:
- 0d2bf63a - Add a container around usage data help-block to prevent horizontal scroll in Firefox
Reassigned to @yorickpeterse
Maybe in a separate MR, but what do we think about moving this data to the License page in the admin area rather than the Settings page?
- It's in a weird spot in the Settings page -- it's somewhat related to the "Version check enabled" setting above it, but completely unrelated to the "Include author name in notification email body" setting below it.
- It will reduce CE-to-EE merge conflicts if the information and implementation are tied to something else that's EE-only.
- With all of this new information, the highlighted JSON payload will be longer and more obstructive in the Settings page, which is already out of control.
Edited by Robert SpeicherAdded 1 commit:
- c2d970f7 - Incorporate review comments
Added 1 commit:
- f179f46c - Simplify test
Added 1 commit:
- 90ee2565 - Remove extraneous pre tag
Added 1 commit:
- 7935a99c - Simplify pre headers
Added 1 commit:
- 4e8aedc1 - Remove inline JavaScript
-
It's in a weird spot in the Settings page -- it's somewhat related to the "Version check enabled" setting above it, but completely unrelated to the "Include author name in notification email body" setting below it. It will reduce CE-to-EE merge conflicts if the information and implementation are tied to something else that's EE-only. With all of this new information, the highlighted JSON payload will be longer and more obstructive in the Settings page, which is already out of control.
Good points. I think there is also talk about having this feature in CE as well (https://gitlab.com/gitlab-org/gitlab-ce/issues/3962), and the license page doesn't exist there. Plus, the setting is right under the "Version check enabled" setting, to which it is related. We could consider moving both of these settings towards the very bottom?
Added 1 commit:
- 863e3790 - Remove inline JavaScript
@stanhu There seem to be some CI failures related to the usage data changes.
Reassigned to @stanhu
Added 1 commit:
- 46b8b680 - Incorporate review comments
Added 1 commit:
- 739e6c43 - Set namespace for Ci modules
Added 1 commit:
- 47d6d33b - Add spec for HistoricalData.max_historical_user_count
Reassigned to @rspeicher
Added 1 commit:
- e2fee92a - Make UsageData a static class again
Maybe in a separate MR, but what do we think about moving this data to the License page in the admin area rather than the Settings page?
- It's in a weird spot in the Settings page -- it's somewhat related to the "Version check enabled" setting above it, but completely unrelated to the "Include author name in notification email body" setting below it.
- It will reduce CE-to-EE merge conflicts if the information and implementation are tied to something else that's EE-only.
@rspeicher @stanhu I rebased and added one field GitLab usage statistics in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5449/diffs before I read your comment!
-
@stanhu So basically, here, at this point, we will send data to version.gitlab.com but will they be recorded? As we have not made any changes on version.gitlab.com yet...
-
@regisF I believe this answers your question https://gitlab.com/gitlab-org/gitlab-ee/issues/997#note_15424030?
-
@axil Just want to make sure that sending more data to version.gitlab.com than what it actually expects to receive won't crash version.gitlab.com
Added 1 commit:
- dcceb1d5 - Use UsageData.to_json as a short-hand
-
Just want to make sure that sending more data to version.gitlab.com than what it actually expects to receive won't crash version.gitlab.com
@regisF I don't think it will, but I do have an out-of-date MR that will support all the new fields: https://dev.gitlab.org/gitlab/version-gitlab-com/merge_requests/36
Added 1 commit:
- 4a0e6d8b - Reduce number of records for HistoricalData spec
Added 1 commit:
- 76a175f2 - Reduce number of records for HistoricalData spec
Added 1 commit:
- 2db914a4 - Add more usage data to EE ping. Load it asynchronously to
Added 1 commit:
- 5067244c - Add more usage data to EE ping.
Enabled an automatic merge when the build for 5067244c succeeds
-
@yorickpeterse @rspeicher Thanks for the thorough review!
Mentioned in commit 63260200
-
I'll submit the docs tomorrow and make the proper changes in the application settings from https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5449.
adding @francisaquino to this issue as an FYI. First step is to begin asking for the data, then for us to collect the info and then to push into other systems, at which point Francis will be involved.
Mentioned in commit 941f7598
-
then for us to collect the info and then to push into other systems, at which point Francis will be involved.
@ChadMalchow we need to talk about this then, because this data is meant to be sent to version.gitlab.com only at this point.
@regisF that is correct and do not want to interrupt this at all. I am having @francisaquino work with someone in engineering to figure out how we can push the data in version.gitlab.com into our CRM tool for customer success and account management to have visibility to take action on and build programs around. This is step 3 and comes after the work you all are doing.
Mentioned in merge request gitlab-com/www-gitlab-com!3202 (merged)
-
@axil once you have the link for the documentation, can you let me know here so I can update my comment for the blog post? Thanks!
