VPN-5092: Help sheet telemetry #9023
Conversation
Request for data collection review form
As a Product Manager I want to know…
This collection is Glean so is documented in the Glean Dictionary.
This collection will be collected permanently.
All channels, countries, and locales. No filters.
These collections are Glean. The opt-out can be found in the product's preferences.
No. |
There was a problem hiding this comment.
r+wc. One note, I think this was a bit overwhelming to review because there is so much new telemetry. In the future, consider breaking the PR down in smaller commits to make it easier to review e.g. metric declaration, implementation and tests for each screen in a different commit.
src/ui/screens/home/ViewServers.qml
Outdated
| }); | ||
| break | ||
| default: | ||
| console.log("Unrecognized selected segment") |
There was a problem hiding this comment.
Consider recording an error value here instead of skipping telemetry. This pattern was adopted for other similar situations. See: https://mozilla-hub.atlassian.net/browse/VPN-5775. If you do it, also update the metric documentation on the YAML file.
There was a problem hiding this comment.
I see how this pattern is used for interaction metrics, which makes sense because the event is still happening, just from an unexpected screen. How would you go about doing something similar for impression metrics where the event doesn't even happen?
There was a problem hiding this comment.
Just do
Glean.impression.locationMultihopScreen.record({
screen: "unexpected"
});
The event is not supposed to happen, but the code path is there so theoretically it can happen. This is just a safe guard, I think it's the same as for interaction metrics at the end of the day.
There was a problem hiding this comment.
Request for data collection review form
1. What questions will you answer with this data?As a Product Manager I want to know…
* how often do users trigger the educational panels whilst visiting the respective app sections? * which sections prompt users to obtain more information by opening the information panel? * does the data indicate that the information on the panels is enough? (expressed another way: Are users needing to click “Learn more” to get the information they were seeking?) * do users who consume the informational panels use the feature in question more than those who don’t? 2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? * This data is necessary for the Mozilla VPN product team to understand the behavior and usage of the educational help sheets and how to make that flow better for users. 3. What alternative methods did you consider to answer these questions? Why were they not sufficient? * None 4. Can current instrumentation answer these questions? * No 5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.Measurement Name Measurement Description Data Collection Category Tracking Bug
impression.app_exclusions_info_screenThe user has opened the app exclusions help sheet interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.dns_settings_info_screenThe user has opened the dns settings help sheet interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.my_devices_screenThe user has opened the devices settings interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.my_devices_info_screenThe user has opened the devices help sheet interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.privacy_features_screenThe user has opened the privacy features settings interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.privacy_features_info_screenThe user has opened the privacy features help sheet interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.location_singlehop_screenhe user has opened to the singlehop screen interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.location_multihop_screenThe user has opened to the multihop screen interaction mozilla-hub.atlassian.net/browse/VPN-5092
impression.location_info_screenThe user has opened the select location help sheet interaction mozilla-hub.atlassian.net/browse/VPN-5092
interaction.help_tooltip_selectedThe user has clicked a help tooltip button interaction mozilla-hub.atlassian.net/browse/VPN-5092
interaction.learn_more_selectedThe user has clicked a learn more link interaction mozilla-hub.atlassian.net/browse/VPN-50926. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.This collection is Glean so is documented in the Glean Dictionary.
7. How long will this data be collected?This collection will be collected permanently. [email protected], [email protected] will be responsible for the permanent collections.
8. What populations will you measure?All channels, countries, and locales. No filters.
9. If this data collection is default on, what is the opt-out mechanism for users?These collections are Glean. The opt-out can be found in the product's preferences.
10. Please provide a general description of how you will analyze this data. * Dashboards that product managers and others will consult on a regular basis. 11. Where do you intend to share the results of your analysis? * Within the Mozilla VPN team and Security and Privacy Products team. 12. Is there a third-party tool (i.e. not Glean or Telemetry) that you are proposing to use for this data collection?No.
Data Review
- Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
Yes, through the metrics.yaml file and the Glean Dictionary.
- Is there a control mechanism that allows the user to turn the data collection on and off?
Yes, through the data preferences in the application settings.
- If the request is for permanent data collection, is there someone who will monitor the data over time?
Permanent collection to be monitored by [email protected], [email protected]
- Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
Category 2, Interaction data
- Is the data collection request for default-on or default-off?
Default-on
- Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?
No
- Is the data collection covered by the existing Firefox privacy notice?
Yes
- Does the data collection use a third-party collection tool?
No
Result
data-review+
Description
Miro board reference: https://miro.com/app/board/uXjVMk7nhvA=/
Needs data review
Reference
VPN-5092: Implement telemetry