Enhance usage ping with product adoption data

We made an EE usage ping in https://gitlab.com/gitlab-org/gitlab-ee/issues/380

https://gitlab.com/gitlab-org/gitlab-ee/uploads/b85639c029c2caa065aa27aa8220ffb3/image.png

If we add more data to this customer success can better help customers.

None of the data should be sensitive (project names, etc.) and everything should be transparent (visible in the JSON string show to the user).

I propose we send the total number of items for:

1. Comments
2. Groups
3. Users
4. Projects
5. Issues
6. Labels
7. CI Jobs or builds
8. Snippets
9. Milestones
10. Todos
11. Pushes
12. Merge requests
13. Environments
14. Triggers
15. Deploy keys
16. Pages
17. Project Services

We should ensure the above are all simple DB queries, if something is complex we shouldn't sent it.

Our version.gitlab.com should do the calculations, such as 'how many comments got made in the last 30 days', and 'did people suddenly stop using something'?

/cc @JobV @stanhu @ChadMalchow

mentioned in issue #847 (closed)

Milestone changed to %8.13

cc @regisF

Can we have the ping data for CE as well? This will help us understand more of what our subscribers are using, both EE and CE, making us more data-driven. We can then make decisions based on data. What we invest resources on building more of. How we better market certain features or products to increase adoption.

I propose that I write a blog post about the CE usage ping and why we want it and then we build it. I think the community reaction will tell us whether it's a good idea.

We'll make it opt-out and make it clear with the upgrade guid.e

@JobV I think that makes a lot of sense, meaning the blog post from you and making it clear that sending us data is optional. I would suggest we think about the messaging to educate users on why we are tracking usage and how we will use the data. Examples, to improve the product, understand what features are being used, improve our marketing around features, help identify new potential features and products.

mentioned in issue #944

Added Product work label

Good idea @JobV.

+1 for collecting data for CE as well.

Notes:

1. Make sure our Privacy policy reflects what we will collect
2. Don't know if there is an impact on the Terms of use

what about gitlab.com? Since we are hosting this, we should be able to collect usage data as well so we can better understand this user group and maybe compare usage as the user got to CE or EE from .com.

At the moment it seems that the ping usage is linked to a license. Is that mandatory? That won't work for CE.

Simple data we can collect (missing from @sytses list):

• the number of issues
• the number of labels

Other point: I get that we shouldn't send heavily computed data. However, here is my take on this. Let's say we decide to improve labels. Apart from feedback we read here and there, we don't know for sure the real usage of labels in the wild. If we wanted to change labels management, we would need answers to those questions first:

• How many labels are there per project?
• How many labels are associated to an issue on average?
• Do merge requests have labels? If so, on many on average?
• Do people use priority labels?
• ...

It's not about measuring everything, because we can easily get lost in metrics and statistics, and waste a huge amount of time on these. But simply knowing the number of labels in total for an account won't nearly help us understand how labels are used.

So it's great to have stats about product adoption. But we definitely need more data than this.

sorry - i think there is an error - why i'm in here ?

@francis my apologies as I meant to add @superfrank316. Feel free to unsubscribe.

I like how Atom is changing both the branding and the UI to ask permissions for their usage collection program.

Jump to Telemetry Permission

https://github.com/atom/atom/releases/tag/v1.11.0-beta0

@JobV @sytses I'd like your input on what I wrote above, as we need to fine tune what we will collect if we want to ship it for 8.13. Collecting more data will help us define a better product, but will also allow to address https://gitlab.com/gitlab-org/gitlab-ee/issues/944 if done well.

@regisF I agree with your comments, but let's start small. Can you make a concrete proposal of something we can ship in 8.13? Make sure to include necessary changes to version.gitlab.com.

For discussion of further data gathering, let's discuss here: https://gitlab.com/gitlab-org/gitlab-ce/issues/21932

@regisF @ChadMalchow @JobV Thanks for the suggestions. My take:

1. We should do the blog post about considering the CE check only after we launched the extended EE check. This will make it easier to reason about for everyone.
2. This check will help us see the activity of GitLab.com
3. We should check our terms and privacy policy and write about this is clear language (link to the documentation).
4. Add the number of issues and the number of labels => great point, I added it to the description
5. Things like 'How many labels are there per project?' => interesting but too computationally intensive, might cause problems, maybe in a later iteration

mentioned in issue gitlab-com/peopleops#94 (closed)

mentioned in issue #997 (closed)

Made a new issue about this feature describing how we will actually do it for 8.13: https://gitlab.com/gitlab-org/gitlab-ee/issues/997

Mentioned in issue #997 (closed)

Mentioned in issue #1024

Status changed to closed

@regisF @JobV now that we launched the extended EE check, when can we do this for CE and blog about it?

Thanks for the follow-up @ChadMalchow

@JobV you said you were willing to write this post. Do you still want to do it?

@regisF @JobV now hat we have begun this for EE, shall we begin collecting data on CE users to help us better understand what features CE users are using and the growth and adoption that is happening in those accounts. With this data we can create and push content to help drive usage of certain features (i.e CI or Issue Boards) to help customers see more value in the product. We can use this data to identify who may find value in EE and push content teaching them what is possible with EE that they may not be aware of.

@ChadMalchow yeah, I think we should: https://gitlab.com/gitlab-org/gitlab-ce/issues/23361

mentioned in issue #1706 (closed)