diff options
Diffstat (limited to '')
| -rw-r--r-- | browser/components/urlbar/docs/contact.rst | 9 | ||||
| -rw-r--r-- | browser/components/urlbar/docs/debugging.rst | 4 | ||||
| -rw-r--r-- | browser/components/urlbar/docs/experiments.rst | 727 | ||||
| -rw-r--r-- | browser/components/urlbar/docs/index.rst | 26 | ||||
| -rw-r--r-- | browser/components/urlbar/docs/overview.rst | 413 | ||||
| -rw-r--r-- | browser/components/urlbar/docs/telemetry.rst | 445 | ||||
| -rw-r--r-- | browser/components/urlbar/docs/utilities.rst | 26 |
7 files changed, 1650 insertions, 0 deletions
diff --git a/browser/components/urlbar/docs/contact.rst b/browser/components/urlbar/docs/contact.rst new file mode 100644 index 0000000000..abd9947528 --- /dev/null +++ b/browser/components/urlbar/docs/contact.rst @@ -0,0 +1,9 @@ +Getting in Touch +================ + +For any questions regarding the Address Bar, the team is available through +the #search channel on Slack and the fx-search@mozilla.com mailing +list. + +Issues can be `filed in Bugzilla <https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=Address%20Bar>`_ +under the Firefox / Address Bar component. diff --git a/browser/components/urlbar/docs/debugging.rst b/browser/components/urlbar/docs/debugging.rst new file mode 100644 index 0000000000..689657c068 --- /dev/null +++ b/browser/components/urlbar/docs/debugging.rst @@ -0,0 +1,4 @@ +Debugging & Logging +=================== + +*Content to be written* diff --git a/browser/components/urlbar/docs/experiments.rst b/browser/components/urlbar/docs/experiments.rst new file mode 100644 index 0000000000..f1217df391 --- /dev/null +++ b/browser/components/urlbar/docs/experiments.rst @@ -0,0 +1,727 @@ +Extensions & Experiments +======================== + +This document describes address bar extensions and experiments: what they are, +how to run them, how to write them, and the processes involved in each. + +The primary purpose right now for writing address bar extensions is to run +address bar experiments. But extensions are useful outside of experiments, and +not all experiments use extensions. + +Like all Firefox extensions, address bar extensions use the WebExtensions_ +framework. + +.. _WebExtensions: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions + +.. toctree:: + :caption: Table of Contents + + experiments + +WebExtensions +------------- + +**WebExtensions** is the name of Firefox's extension architecture. The "web" +part of the name hints at the fact that Firefox extensions are built using Web +technologies: JavaScript, HTML, CSS, and to a certain extent the DOM. + +Individual extensions themselves often are referred to as *WebExtensions*. For +clarity and conciseness, this document will refer to WebExtensions as +*extensions*. + +Why are we interested in extensions? Mainly because they're a powerful way to +run experiments in Firefox. See Experiments_ for more on that. In addition, we'd +also like to build up a robust set of APIs useful to extension authors, although +right now the API can only be used by Mozilla extensions. + +WebExtensions are introduced and discussed in detail on `MDN +<WebExtensions_>`__. You'll need a lot of that knowledge in order to build +address bar extensions. + +Developing Address Bar Extensions +--------------------------------- + +Overview +~~~~~~~~ + +The address bar WebExtensions API currently lives in two API namespaces, +``browser.urlbar`` and ``browser.experiments.urlbar``. The reason for this is +historical and is discussed in the `Developing Address Bar Extension APIs`_ +section. As a consumer of the API, there are only two important things you need +to know: + +* There's no meaningful difference between the APIs of the two namespaces. + Their kinds of functions, events, and properties are similar. You should + think of the address bar API as one single API that happens to be split into + two namespaces. + +* However, there is a big difference between the two when it comes to setting up + your extension to use them. This is discussed next. + +The ``browser.urlbar`` API namespace is built into Firefox. It's a +**privileged API**, which means that only Mozilla-signed and temporarily +installed extensions can use it. The only thing your Mozilla extension needs to +do in order to use it is to request the ``urlbar`` permission in its +manifest.json, as illustrated `here <urlbarPermissionExample_>`__. + +In contrast, the ``browser.experiments.urlbar`` API namespace is bundled inside +your extension. APIs that are bundled inside extensions are called +**experimental APIs**, and the extensions in which they're bundled are called +**WebExtension experiments**. As with privileged APIs, experimental APIs are +available only to Mozilla-signed and temporarily installed extensions. +("WebExtension experiments" is a term of art and shouldn't be confused with the +general notion of experiments that happen to use extensions.) For more on +experimental APIs and WebExtension experiments, see the `WebExtensions API +implementation documentation <webextAPIImplBasicsDoc_>`__. + +Since ``browser.experiments.urlbar`` is bundled inside your extension, you'll +need to include it in your extension's repo by doing the following: + +1. The implementation consists of two files, api.js_ and schema.json_. In your + extension repo, create a *experiments/urlbar* subdirectory and copy the + files there. See `this repo`__ for an example. + +2. Add the following ``experiment_apis`` key to your manifest.json (see here__ + for an example in context):: + + "experiment_apis": { + "experiments_urlbar": { + "schema": "experiments/urlbar/schema.json", + "parent": { + "scopes": ["addon_parent"], + "paths": [["experiments", "urlbar"]], + "script": "experiments/urlbar/api.js" + } + } + } + +As mentioned, only Mozilla-signed and temporarily installed extensions can use +these two API namespaces. For information on running the extensions you develop +that use these namespaces, see `Running Address Bar Extensions`_. + +.. _urlbarPermissionExample: https://github.com/0c0w3/urlbar-top-sites-experiment/blob/ac1517118bb7ee165fb9989834514b1082575c10/src/manifest.json#L24 +.. _webextAPIImplBasicsDoc: https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/basics.html +.. _api.js: https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/api.js +.. _schema.json: https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/schema.json +__ https://github.com/0c0w3/dynamic-result-type-extension/tree/master/src/experiments/urlbar +__ https://github.com/0c0w3/dynamic-result-type-extension/blob/0987da4b259b9fcb139b31d771883a2f822712b5/src/manifest.json#L28 + +browser.urlbar +~~~~~~~~~~~~~~ + +Currently the only documentation for ``browser.urlbar`` is its `schema +<urlbar.json_>`__. Fortunately WebExtension schemas are JSON and aren't too hard +to read. If you need help understanding it, see the `WebExtensions API +implementation documentation <webextAPIImplDoc_>`__. + +For examples on using the API, see the Cookbook_ section. + +.. _urlbar.json: https://searchfox.org/mozilla-central/source/browser/components/extensions/schemas/urlbar.json + +browser.experiments.urlbar +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As with ``browser.urlbar``, currently the only documentation for +``browser.experiments.urlbar`` is its schema__. For examples on using the API, +see the Cookbook_ section. + +__ https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/schema.json + +Workflow +~~~~~~~~ + +The web-ext_ command-line tool makes the extension-development workflow very +simple. Simply start it with the *run* command, passing it the location of the +Firefox binary you want to use. web-ext will launch your Firefox and remain +running until you stop it, watching for changes you make to your extension's +files. When it sees a change, it automatically reloads your extension — in +Firefox, in the background — without your having to do anything. It's really +nice. + +The `web-ext documentation <web-ext commands_>`__ lists all its options, but +here are some worth calling out for the *run* command: + +``--browser-console`` + Automatically open the browser console when Firefox starts. Very useful for + watching your extension's console logging. (Make sure "Show Content Messages" + is checked in the console.) + +``-p`` + This option lets you specify a path to a profile directory. + +``--keep-profile-changes`` + Normally web-ext doesn't save any changes you make to the profile. Use this + option along with ``-p`` to reuse the same profile again and again. + +``--verbose`` + web-ext suppresses Firefox messages in the terminal unless you pass this + option. If you've added some ``dump`` calls in Firefox because you're working + on a new ``browser.urlbar`` API, for example, you won't see them without this. + +web-ext also has a *build* command that packages your extension's files into a +zip file. The following *build* options are useful: + +``--overwrite-dest`` + Without this option, web-ext won't overwrite a zip file it previously created. + +web-ext can load its configuration from your extension's package.json. That's +the recommended way to configure it. Here's an example__. + +Finally, web-ext can also sign extensions, but if you're developing your +extension for an experiment, you'll use a different process for signing. See +`The Experiment Development Process`_. + +.. _web-ext: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Getting_started_with_web-ext +.. _web-ext commands: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/web-ext_command_reference +__ https://github.com/0c0w3/urlbar-top-sites-experiment/blob/6681a7126986bc2565d036b888cb5b8807397ce5/package.json#L7 + +Automated Tests +~~~~~~~~~~~~~~~ + +It's possible to write `browser chrome mochitests`_ for your extension the same +way we write tests for Firefox. One of the example extensions linked throughout +this document includes a test_, for instance. + +See the readme in the example-addon-experiment_ repo for a workflow. + +.. _browser chrome mochitests: https://developer.mozilla.org/en-US/docs/Mozilla/Browser_chrome_tests +.. _test: https://github.com/0c0w3/urlbar-top-sites-experiment/blob/master/tests/tests/browser/browser_urlbarTopSitesExtension.js + +Cookbook +~~~~~~~~ + +*To be written.* For now, you can find example uses of ``browser.experiments.urlbar`` and ``browser.urlbar`` in the following repos: + +* https://github.com/mozilla-extensions/firefox-quick-suggest-weather +* https://github.com/0c0w3/urlbar-tips-experiment +* https://github.com/0c0w3/urlbar-top-sites-experiment +* https://github.com/0c0w3/urlbar-search-interventions-experiment + +Further Reading +~~~~~~~~~~~~~~~ + +`WebExtensions on MDN <WebExtensions_>`__ + The place to learn about developing WebExtensions in general. + +`Getting started with web-ext <web-ext_>`__ + MDN's tutorial on using web-ext. + +`web-ext command reference <web-ext commands_>`__ + MDN's documentation on web-ext's commands and their options. + +Developing Address Bar Extension APIs +------------------------------------- + +Built-In APIs vs. Experimental APIs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Originally we developed the address bar extension API in the ``browser.urlbar`` +namespace, which is built into Firefox as discussed above. By "built into +Firefox," we mean that the API is developed in `mozilla-central +<urlbar.json_>`__ and shipped inside Firefox just like any other Firefox +feature. At the time, that seemed like the right thing to do because we wanted +to build an API that ultimately could be used by all extension authors, not only +Mozilla. + +However, there were a number of disadvantages to this development model. The +biggest was that it tightly coupled our experiments to specific versions of +Firefox. For example, if we were working on an experiment that targeted Firefox +72, then any APIs used by that experiment needed to land and ship in 72. If we +weren't able to finish an API by the time 72 shipped, then the experiment would +have to be postponed until 73. Our experiment development timeframes were always +very short because we always wanted to ship our experiments ASAP. Often we +targeted the Firefox version that was then in Nightly; sometimes we even +targeted the version in Beta. Either way, it meant that we were always uplifting +patch after patch to Beta. This tight coupling between Firefox versions and +experiments erased what should have been a big advantage of implementing +experiments as extensions in the first place: the ability to ship experiments +outside the usual cyclical release process. + +Another notable disadvantage of this model was just the cognitive weight of the +idea that we were developing APIs not only for ourselves and our experiments but +potentially for all extensions. This meant that not only did we have to design +APIs to meet our immediate needs, we also had to imagine use cases that could +potentially arise and then design for them as well. + +For these reasons, we stopped developing ``browser.urlbar`` and created the +``browser.experiments.urlbar`` experimental API. As discussed earlier, +experimental APIs are APIs that are bundled inside extensions. Experimental APIs +can do anything that built-in APIs can do with the added flexibility of not +being tied to specific versions of Firefox. + +Adding New APIs +~~~~~~~~~~~~~~~ + +All new address bar APIs should be added to ``browser.experiments.urlbar``. +Although this API does not ship in Firefox, it's currently developed in +mozilla-central, in `browser/components/urlbar/tests/ext/ <extDirectory_>`__ -- +note the "tests" subdirectory. Developing it in mozilla-central lets us take +advantage of our usual build and testing infrastructure. This way we have API +tests running against each mozilla-central checkin, against all versions of +Firefox that are tested on Mozilla's infrastructure, and we're alerted to any +breaking changes we accidentally make. When we start a new extension repo, we +copy schema.json and api.js to it as described earlier (or clone an example repo +with up-to-date copies of these files). + +Generally changes to the API should be reviewed by someone on the address bar +team and someone on the WebExtensions team. Shane (mixedpuppy) is a good +contact. + +.. _extDirectory: https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/ + +Anatomy of an API +~~~~~~~~~~~~~~~~~ + +Roughly speaking, a WebExtensions API implementation comprises three different +pieces: + +Schema + The schema declares the functions, properties, events, and types that the API + makes available to extensions. Schemas are written in JSON. + + The ``browser.experiments.urlbar`` schema is schema.json_, and the + ``browser.urlbar`` schema is urlbar.json_. + + For reference, the schemas of built-in APIs are in + `browser/components/extensions/schemas`_ and + `toolkit/components/extensions/schemas`_. + + .. _browser/components/extensions/schemas: https://searchfox.org/mozilla-central/source/browser/components/extensions/schemas/ + .. _toolkit/components/extensions/schemas: https://searchfox.org/mozilla-central/source/toolkit/components/extensions/schemas/ + +Internals + Every API hooks into some internal part of Firefox. For the address bar API, + that's the Urlbar implementation in `browser/components/urlbar`_. + + .. _browser/components/urlbar: https://searchfox.org/mozilla-central/source/browser/components/urlbar/ + +Glue + Finally, there's some glue code that implements everything declared in the + schema. Essentially, this code mediates between the previous two pieces. It + translates the function calls, property accesses, and event listener + registrations made by extensions using the public-facing API into terms that + the Firefox internals understand, and vice versa. + + For ``browser.experiments.urlbar``, this is api.js_, and for + ``browser.urlbar``, it's ext-urlbar.js_. + + For reference, the implementations of built-in APIs are in + `browser/components/extensions`_ and `toolkit/components/extensions`_, in the + *parent* and *child* subdirecties. As you might guess, code in *parent* runs + in the main process, and code in *child* runs in the extensions process. + Address bar APIs deal with browser chrome and their implementations therefore + run in the parent process. + + .. _ext-urlbar.js: https://searchfox.org/mozilla-central/source/browser/components/extensions/parent/ext-urlbar.js + .. _browser/components/extensions: https://searchfox.org/mozilla-central/source/browser/components/extensions/ + .. _toolkit/components/extensions: https://searchfox.org/mozilla-central/source/toolkit/components/extensions/ + +Keep in mind that extensions run in a different process from the main process. +That has implications for your APIs. They'll generally need to be async, for +example. + +Further Reading +~~~~~~~~~~~~~~~ + +`WebExtensions API implementation documentation <webextAPIImplDoc_>`__ + Detailed info on implementing a WebExtensions API. + +.. _webextAPIImplDoc: https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/ + +Running Address Bar Extensions +------------------------------ + +As discussed above, ``browser.experiments.urlbar`` and ``browser.urlbar`` are +privileged APIs. There are two different points to consider when it comes to +running an extension that uses privileged APIs: loading the extension in the +first place, and granting it access to privileged APIs. There's a certain bar +for loading any extension regardless of its API usage that depends on its signed +state and the Firefox build you want to run it in. There's yet a higher bar for +granting it access to privileged APIs. This section discusses how to load +extensions so that they can access privileged APIs. + +Since we're interested in extensions primarily for running experiments, there +are three particular signed states relevant to us: + +Unsigned + There are two ways to run unsigned extensions that use privileged APIs. + + They can be loaded temporarily using a Firefox Nightly build or + Developer Edition but not Beta or Release [source__], and the + ``extensions.experiments.enabled`` preference must be set to true [source__]. + You can load extensions temporarily by visiting + about:debugging#/runtime/this-firefox and clicking "Load Temporary Add-on." + `web-ext <Workflow_>`__ also loads extensions temporarily. + + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/components/extensions/Extension.jsm#1884 + __ https://searchfox.org/mozilla-central/rev/014fe72eaba26dcf6082fb9bbaf208f97a38594e/toolkit/mozapps/extensions/internal/AddonSettings.jsm#93 + + They can be also be loaded normally (not temporarily) in a custom build where + the build-time setting ``AppConstants.MOZ_REQUIRE_SIGNING`` [source__, source__] + and ``xpinstall.signatures.required`` pref are both false. As in the previous + paragraph, such builds include Nightly and Developer Edition but not Beta or + Release [source__]. In addition, your custom build must modify the + ``Extension.isPrivileged`` getter__ to return true. This getter determines + whether an extension can access privileged APIs. + + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/mozapps/extensions/internal/XPIProvider.jsm#2382 + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/mozapps/extensions/internal/AddonSettings.jsm#36 + __ https://searchfox.org/mozilla-central/search?q=MOZ_REQUIRE_SIGNING&case=false®exp=false&path= + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/components/extensions/Extension.jsm#1874 + + Extensions remain unsigned as you develop them. See the Workflow_ section for + more. + +Signed for testing (Signed for QA) + Signed-for-testing extensions that use privileged APIs can be run using the + same techniques for running unsigned extensions. + + They can also be loaded normally (not temporarily) if you use a Firefox build + where the build-time setting ``AppConstants.MOZ_REQUIRE_SIGNING`` is false and + you set the ``xpinstall.signatures.dev-root`` pref to true + [source__]. ``xpinstall.signatures.dev-root`` does not exist by default and + must be created. + + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/mozapps/extensions/internal/XPIInstall.jsm#262 + + You encounter extensions that are signed for testing when you are writing + extensions for experiments. See the Experiments_ section for details. + + "Signed for QA" is another way of referring to this signed state. + +Signed for release + Signed-for-release extensions that use privileged APIs can be run in any + Firefox build with no special requirements. + + You encounter extensions that are signed for release when you are writing + extensions for experiments. See the Experiments_ section for details. + +.. important:: + To see console logs from extensions in the browser console, select the "Show + Content Messages" option in the console's settings. This is necessary because + extensions run outside the main process. + +Experiments +----------- + +**Experiments** let us try out ideas in Firefox outside the usual release cycle +and on particular populations of users. + +For example, say we have a hunch that the top sites shown on the new-tab page +aren't very discoverable, so we want to make them more visible. We have one idea +that might work — show them every time the user begins an interaction with the +address bar — but we aren't sure how good an idea it is. So we test it. We write +an extension that does just that, make sure it collects telemetry that will help +us answer our question, ship it outside the usual release cycle to a small +percentage of Beta users, collect and analyze the telemetry, and determine +whether the experiment was successful. If it was, then we might want to ship the +feature to all Firefox users. + +Experiments sometimes are also called **studies** (not to be confused with *user +studies*, which are face-to-face interviews with users conducted by user +researchers). + +There are two types of experiments: + +Pref-flip experiments + Pref-flip experiments are simple. If we have a fully baked feature in the + browser that's preffed off, a pref-flip experiment just flips the pref on, + enabling the feature for users running the experiment. No code is required. + We tell the experiments team the name of the pref we want to flip, and they + handle it. + + One important caveat to pref-flip studies is that they're currently capable of + flipping only a single pref. There's an extension called Multipreffer_ that + can flip multiple prefs, though. + + .. _Multipreffer: https://github.com/mozilla/multipreffer + +Add-on experiments + Add-on experiments are much more complex but much more powerful. (Here + *add-on* is a synonym for extension.) They're the type of experiments that + this document has been discussing all along. + + An add-on experiment is shipped as an extension that we write and that + implements the experimental feature we want to test. To reiterate, the + extension is a WebExtension and uses WebExtensions APIs. If the current + WebExtensions APIs do not meet the needs of your experiment, then you must + create either experimental or built-in APIs so that your extension can use + them. If necessary, you can make any new built-in APIs privileged so that they + are available only to Mozilla extensions. + + An add-on experiment can collect additional telemetry that's not collected in + the product by using the privileged ``browser.telemetry`` WebExtensions API, + and of course the product will continue to collect all the telemetry it + usually does. The telemetry pings from users running the experiment will be + correlated with the experiment with no extra work on our part. + +A single experiment can deliver different UXes to different groups of users +running the experiment. Each group or UX within an experiment is called a +**branch**. Experiments often have two branches, control and treatment. The +**control branch** actually makes no UX changes. It may capture additional +telemetry, though. Think of it as the control in a science experiment. It's +there so we can compare it to data from the **treatment branch**, which does +make UX changes. Some experiments may require multiple treatment branches, in +which case the different branches will have different names. Add-on experiments +can implement all branches in the same extension or each branch in its own +extension. + +Experiments are delivered to users by a system called **Normandy**. Normandy +comprises a client side that lives in Firefox and a server side. In Normandy, +experiments are defined server-side in files called **recipes**. Recipes include +information about the experiment like the Firefox release channel and version +that the experiment targets, the number of users to be included in the +experiment, the branches in the experiment, the percentage of users on each +branch, and so on. + +Experiments are tracked by Mozilla project management using a system called +Experimenter_. + +Finally, there was an older version of the experiments program called +**Shield**. Experiments under this system were called **Shield studies** and +could be be shipped as extensions too. + +.. _Experimenter: https://experimenter.services.mozilla.com/ + +Further Reading +~~~~~~~~~~~~~~~ + +`Pref-Flip and Add-On Experiments <https://mana.mozilla.org/wiki/pages/viewpage.action?spaceKey=FIREFOX&title=Pref-Flip+and+Add-On+Experiments>`__ + A comprehensive document on experiments from the Experimenter team. See the + child pages in the sidebar, too. + +`Client Implementation Guidelines for Experiments <https://docs.telemetry.mozilla.org/cookbooks/client_guidelines.html>`_ + Relevant documentation from the telemetry team. + +#ask-experimenter Slack channel + A friendly place to get answers to your experiment questions. + +The Experiment Development Process +---------------------------------- + +This section describes an experiment's life cycle. + +1. Experiments usually originate with product management and UX. They're + responsible for identifying a problem, deciding how an experiment should + approach it, the questions we want to answer, the data we need to answer + those questions, the user population that should be enrolled in the + experiment, the definition of success, and so on. + +2. UX makes a spec that describes what the extension looks like and how it + behaves. + +3. There's a kickoff meeting among the team to introduce the experiment and UX + spec. It's an opportunity for engineering to ask questions of management, UX, + and data science. It's really important for engineering to get a precise and + accurate understanding of how the extension is supposed to behave — right + down to the UI changes — so that no one makes erroneous assumptions during + development. + +4. At some point around this time, the team (usually management) creates a few + artifacts to track the work and facilitate communication with outside teams + involved in shipping experiments. They include: + + * A page on `Experimenter <Experiments_>`__ + * A QA PI (product integrity) request so that QA resources are allocated + * A bug in `Data Science :: Experiment Collaboration`__ so that data science + can track the work and discuss telemetry (engineering might file this one) + + __ https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_ignored=0&bug_severity=normal&bug_status=NEW&bug_type=task&cf_firefox_messaging_system=---&cf_fx_iteration=---&cf_fx_points=---&comment=%23%23%20Brief%20Description%20of%20the%20request%20%28required%29%3A%0D%0A%0D%0A%23%23%20Business%20purpose%20for%20this%20request%20%28required%29%3A%0D%0A%0D%0A%23%23%20Requested%20timelines%20for%20the%20request%20or%20how%20this%20fits%20into%20roadmaps%20or%20critical%20decisions%20%28required%29%3A%0D%0A%0D%0A%23%23%20Links%20to%20any%20assets%20%28e.g%20Start%20of%20a%20PHD%2C%20BRD%3B%20any%20document%20that%20helps%20describe%20the%20project%29%3A%0D%0A%0D%0A%23%23%20Name%20of%20Data%20Scientist%20%28If%20Applicable%29%3A%0D%0A%0D%0A%2APlease%20note%20if%20it%20is%20found%20that%20not%20enough%20information%20has%20been%20given%20this%20will%20delay%20the%20triage%20of%20this%20request.%2A&component=Experiment%20Collaboration&contenttypemethod=list&contenttypeselection=text%2Fplain&filed_via=standard_form&flag_type-4=X&flag_type-607=X&flag_type-800=X&flag_type-803=X&flag_type-936=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Data%20Science&rep_platform=Unspecified&target_milestone=---&version=unspecified + +5. Engineering breaks down the work and files bugs. There's another engineering + meeting to discuss the breakdown, or it's discussed asynchronously. + +6. Engineering sets up a GitHub repo for the extension. See `Implementing + Experiments`_ for an example repo you can clone to get started. Disable + GitHub Issues on the repo so that QA will file bugs in Bugzilla instead of + GitHub. There's nothing wrong with GitHub Issues, but our team's project + management tracks all work through Bugzilla. If it's not there, it's not + captured. + +7. Engineering or management fills out the Add-on section of the Experimenter + page as much as possible at this point. "Active Experiment Name" isn't + necessary, and "Signed Release URL" won't be available until the end of the + process. + +8. Engineering implements the extension and any new WebExtensions APIs it + requires. + +9. When the extension is done, engineering or management clicks the "Ready for + Sign-Off" button on the Experimenter page. That changes the page's status + from "Draft" to "Ready for Sign-Off," which allows QA and other teams to sign + off on their portions of the experiment. + +10. Engineering requests the extension be signed "for testing" (or "for + QA"). Michael (mythmon) from the Experiments team and Rehan (rdalal) from + Services Engineering are good contacts. Build the extension zip file using + web-ext as discussed in Workflow_. Attach it to a bug (a metabug for + implementing the extension, for example), needinfo Michael or Rehan, and ask + him to sign it. He'll attach the signed version to the bug. If neither + Michael nor Rehan is available, try asking in the #ask-experimenter Slack + channel. + +11. Engineering sends QA the link to the signed extension and works with them to + resolve bugs they find. + +12. When QA signs off, engineering asks Michael to sign the extension "for + release" using the same needinfo process described earlier. + +13. Paste the URL of the signed extension in the "Signed Release URL" textbox of + the Add-on section of the Experimenter page. + +14. Other teams sign off as they're ready. + +15. The experiment ships! 🎉 + + +Implementing Experiments +------------------------ + +This section discusses how to implement add-on experiments. Pref-flip +experiments are much simpler and don't need a lot of explanation. You should be +familiar with the concepts discussed in the `Developing Address Bar Extensions`_ +and `Running Address Bar Extensions`_ sections before reading this one. + +The most salient thing about add-on experiments is that they're implemented +simply as privileged extensions. Other than being privileged and possibly +containing bundled experimental APIs, they're similar to all other extensions. + +The `top-sites experiment extension <topSites_>`__ is an example of a real, +shipped experiment. + +.. _topSites: https://github.com/0c0w3/urlbar-top-sites-experiment + +Setup +~~~~~ + +example-addon-experiment_ is a repo you can clone to get started. It's geared +toward urlbar extensions and includes the stub of a browser chrome mochitest. + +.. _example-addon-experiment: https://github.com/0c0w3/example-addon-experiment + +browser.normandyAddonStudy +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As discussed in Experiments_, an experiment typically has more than one branch +so that it can test different UXes. The experiment's extension(s) needs to know +the branch the user is enrolled in so that it can behave appropriately for the +branch: show the user the proper UX, collect the proper telemetry, and so on. + +This is the purpose of the ``browser.normandyAddonStudy`` WebExtensions API. +Like ``browser.urlbar``, it's a privileged API available only to Mozilla +extensions. + +Its schema is normandyAddonStudy.json_. + +It's a very simple API. The primary function is ``getStudy``, which returns the +study the user is currently enrolled in or null if there isn't one. (Recall that +*study* is a synonym for *experiment*.) One of the first things an experiment +extension typically does is to call this function. + +The Normandy client in Firefox will keep an experiment extension installed only +while the experiment is active. Therefore, ``getStudy`` should always return a +non-null study object. Nevertheless, the study object has an ``active`` boolean +property that's trivial to sanity check. (The example extension does.) + +The more important property is ``branch``, the name of the branch that the user +is enrolled in. Your extension should use it to determine the appropriate UX. + +Finally, there's an ``onUnenroll`` event that's fired when the user is +unenrolled in the study. It's not quite clear in what cases an extension would +need to listen for this event given that Normandy automatically uninstalls +extensions on unenrollment. Maybe if they create some persistent state that's +not automatically undone on uninstall by the WebExtensions framework? + +If your extension itself needs to unenroll the user for some reason, call +``endStudy``. + +.. _normandyAddonStudy.json: https://searchfox.org/mozilla-central/source/browser/components/extensions/schemas/normandyAddonStudy.json + +Telemetry +~~~~~~~~~ + +Experiments can capture telemetry in two places: in the product itself and +through the privileged ``browser.telemetry`` WebExtensions API. The API schema +is telemetry.json_. + +The telemetry pings from users running experiments are automatically correlated +with those experiments, no extra work required. That's true regardless of +whether the telemetry is captured in the product or though +``browser.telemetry``. + +The address bar has some in-product, preffed off telemetry that we want to +enable for all our experiments — at least that's the thinking as of August 2019. +It's called `engagement event telemetry`_, and it records user *engagements* +with and *abandonments* of the address bar [source__]. We added a +BrowserSetting_ on ``browser.urlbar`` just to let us flip the pref and enable +this telemetry in our experiment extensions. Call it like this:: + + await browser.urlbar.engagementTelemetry.set({ value: true }); + +.. _telemetry.json: https://searchfox.org/mozilla-central/source/toolkit/components/extensions/schemas/telemetry.json +.. _engagement event telemetry: https://bugzilla.mozilla.org/show_bug.cgi?id=1559136 +__ https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/browser/components/urlbar/UrlbarController.jsm#598 +.. _BrowserSetting: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/types/BrowserSetting + +Engineering Best Practices +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Clear up questions with your UX person early and often. There's often a gap +between what they have in their mind and what you have in yours. Nothing wrong +with that, it's just the nature of development. But misunderstandings can cause +big problems when they're discovered late. This is especially true of UX +behaviors, as opposed to visuals or styling. It's no fun to realize at the end +of a release cycle that you've designed the wrong WebExtensions API because some +UX detail was overlooked. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Related to the previous point, make builds of your extension for your UX person +so they can test it. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Taking the previous point even further, if your experiment will require a +substantial new API(s), you might think about prototyping the experiment +entirely in a custom Firefox build before designing the API at all. Give it to +your UX person. Let them disect it and tell you all the problems with it. Fill +in all the gaps in your understanding, and then design the API. We've never +actually done this, though. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It's a good idea to work on the extension as you're designing and developing the +APIs it'll use. You might even go as far as writing the first draft of the +extension before even starting to implement the APIs. That lets you spot +problems that may not be obvious were you to design the API in isolation. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Your extension's ID should end in ``@shield.mozilla.org``. QA will flag it if it +doesn't. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set ``"hidden": true`` in your extension's manifest.json. That hides it on +about:addons. (It can still be seen on about:studies.) QA will spot this if you +don't. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are drawbacks of hiding features behind prefs and enabling them in +experiment extensions. Consider not doing that if feasible, or at least weigh +these drawbacks against your expected benefits. + +* Prefs stay flipped on in private windows, but experiments often have special + requirements around private-browsing mode (PBM). Usually, they shouldn't be + active in PBM at all, unless of course the point of the experiment is to test + PBM. Extensions also must request PBM access ("incognito" in WebExtensions + terms), and the user can disable access at any time. The result is that part + of your experiment could remain enabled — the part behind the pref — while + other parts are disabled. + +* Prefs stay flipped on in safe mode, even though your extension (like all + extensions) will be disabled. This might be a bug__ in the WebExtensions + framework, though. + + __ https://bugzilla.mozilla.org/show_bug.cgi?id=1576997 diff --git a/browser/components/urlbar/docs/index.rst b/browser/components/urlbar/docs/index.rst new file mode 100644 index 0000000000..633bca91b6 --- /dev/null +++ b/browser/components/urlbar/docs/index.rst @@ -0,0 +1,26 @@ +Address Bar +=========== + +This document describes the implementation of Firefox's address bar, also known +as the quantumbar or urlbar. The address bar was also called the awesomebar +until Firefox 68, when it was substantially rewritten. + +The address bar is a specialized search access point that aggregates data from +several different sources, including: + + * Places (Firefox's history and bookmarks system) + * Search engines (including search suggestions) + * WebExtensions + * Open tabs + +Most of the address bar code lives in `browser/components/urlbar <https://searchfox.org/mozilla-central/source/browser/components/urlbar/>`_. +A separate and important back-end piece currently is `toolkit/components/places/UnifiedComplete.jsm <https://searchfox.org/mozilla-central/source/toolkit/components/places/UnifiedComplete.jsm>`_, which was carried over from awesomebar and has not yet been rewritten for quantumbar. + +.. toctree:: + + overview + utilities + telemetry + debugging + experiments + contact diff --git a/browser/components/urlbar/docs/overview.rst b/browser/components/urlbar/docs/overview.rst new file mode 100644 index 0000000000..e7306bceea --- /dev/null +++ b/browser/components/urlbar/docs/overview.rst @@ -0,0 +1,413 @@ +Architecture Overview +===================== + +The address bar is implemented as a *model-view-controller* (MVC) system. One of +the scopes of this architecture is to allow easy replacement of its components, +for easier experimentation. + +Each search is represented by a unique object, the *UrlbarQueryContext*. This +object, created by the *View*, describes the search and is passed through all of +the components, along the way it gets augmented with additional information. +The *UrlbarQueryContext* is passed to the *Controller*, and finally to the +*Model*. The model appends results to a property of *UrlbarQueryContext* in +chunks, it sorts them through a *Muxer* and then notifies the *Controller*. + +See the specific components below, for additional details about each one's tasks +and responsibilities. + + +The UrlbarQueryContext +---------------------- + +The *UrlbarQueryContext* object describes a single instance of a search. +It is augmented as it progresses through the system, with various information: + +.. highlight:: JavaScript +.. code:: + + UrlbarQueryContext { + allowAutofill; // {boolean} If true, providers are allowed to return + // autofill results. Even if true, it's up to providers + // whether to include autofill results, but when false, no + // provider should include them. + isPrivate; // {boolean} Whether the search started in a private context. + maxResults; // {integer} The maximum number of results requested. It is + // possible to request more results than the shown ones, and + // do additional filtering at the View level. + searchString; // {string} The user typed string. + userContextId; // {integer} The user context ID (containers feature). + + // Optional properties. + muxer; // {string} Name of a registered muxer. Muxers can be registered + // through the UrlbarProvidersManager. + providers; // {array} List of registered provider names. Providers can be + // registered through the UrlbarProvidersManager. + sources: {array} list of accepted UrlbarUtils.RESULT_SOURCE for the context. + // This allows to switch between different search modes. If not + // provided, a default will be generated by the Model, depending on + // the search string. + engineName: // {string} if sources is restricting to just SEARCH, this + // property can be used to pick a specific search engine, by + // setting it to the name under which the engine is registered + // with the search service. + currentPage: // {string} url of the page that was loaded when the search + // began. + allowSearchSuggestions: // {boolean} Whether to allow search suggestions. + // This is a veto, meaning that when false, + // suggestions will not be fetched, but when true, + // some other condition may still prohibit + // suggestions, like private browsing mode. Defaults + // to true. + + // Properties added by the Model. + results; // {array} list of UrlbarResult objects. + tokens; // {array} tokens extracted from the searchString, each token is an + // object in the form {type, value, lowerCaseValue}. + } + + +The Model +--------- + +The *Model* is the component responsible for retrieving search results based on +the user's input, and sorting them accordingly to their importance. +At the core is the `UrlbarProvidersManager <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarProvidersManager.jsm>`_, +a component tracking all the available search providers, and managing searches +across them. + +The *UrlbarProvidersManager* is a singleton, it registers internal providers on +startup and can register/unregister providers on the fly. +It can manage multiple concurrent queries, and tracks them internally as +separate *Query* objects. + +The *Controller* starts and stops queries through the *UrlbarProvidersManager*. +It's possible to wait for the promise returned by *startQuery* to know when no +more results will be returned, it is not mandatory though. +Queries can be canceled. + +.. note:: + + Canceling a query will issue an interrupt() on the database connection, + terminating any running and future SQL query, unless a query is running inside + a *runInCriticalSection* task. + +The *searchString* gets tokenized by the `UrlbarTokenizer <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarTokenizer.jsm>`_ +component into tokens, some of these tokens have a special meaning and can be +used by the user to restrict the search to specific result type (See the +*UrlbarTokenizer::TYPE* enum). + +.. caution:: + + The tokenizer uses heuristics to determine each token's type, as such the + consumer may want to check the value before applying filters. + +.. highlight:: JavaScript +.. code:: + + UrlbarProvidersManager { + registerProvider(providerObj); + unregisterProvider(providerObj); + registerMuxer(muxerObj); + unregisterMuxer(muxerObjOrName); + async startQuery(queryContext); + cancelQuery(queryContext); + // Can be used by providers to run uninterruptible queries. + runInCriticalSection(taskFn); + } + +UrlbarProvider +~~~~~~~~~~~~~~ + +A provider is specialized into searching and returning results from different +information sources. Internal providers are usually implemented in separate +*jsm* modules with a *UrlbarProvider* name prefix. External providers can be +registered as *Objects* through the *UrlbarProvidersManager*. +Each provider is independent and must satisfy a base API, while internal +implementation details may vary deeply among different providers. + +.. important:: + + Providers are singleton, and must track concurrent searches internally, for + example mapping them by UrlbarQueryContext. + +.. note:: + + Internal providers can access the Places database through the + *PlacesUtils.promiseLargeCacheDBConnection* utility. + +.. highlight:: JavaScript +.. code:: + + class UrlbarProvider { + /** + * Unique name for the provider, used by the context to filter on providers. + * Not using a unique name will cause the newest registration to win. + * @abstract + */ + get name() { + return "UrlbarProviderBase"; + } + /** + * The type of the provider, must be one of UrlbarUtils.PROVIDER_TYPE. + * @abstract + */ + get type() { + throw new Error("Trying to access the base class, must be overridden"); + } + /** + * Whether this provider should be invoked for the given context. + * If this method returns false, the providers manager won't start a query + * with this provider, to save on resources. + * @param {UrlbarQueryContext} queryContext The query context object + * @returns {boolean} Whether this provider should be invoked for the search. + * @abstract + */ + isActive(queryContext) { + throw new Error("Trying to access the base class, must be overridden"); + } + /** + * Gets the provider's priority. Priorities are numeric values starting at + * zero and increasing in value. Smaller values are lower priorities, and + * larger values are higher priorities. For a given query, `startQuery` is + * called on only the active and highest-priority providers. + * @param {UrlbarQueryContext} queryContext The query context object + * @returns {number} The provider's priority for the given query. + * @abstract + */ + getPriority(queryContext) { + // By default, all providers share the lowest priority. + return 0; + } + /** + * Starts querying. + * @param {UrlbarQueryContext} queryContext The query context object + * @param {function} addCallback Callback invoked by the provider to add a new + * result. A UrlbarResult should be passed to it. + * @note Extended classes should return a Promise resolved when the provider + * is done searching AND returning results. + * @abstract + */ + startQuery(queryContext, addCallback) { + throw new Error("Trying to access the base class, must be overridden"); + } + /** + * Cancels a running query, + * @param {UrlbarQueryContext} queryContext The query context object to cancel + * query for. + * @abstract + */ + cancelQuery(queryContext) { + throw new Error("Trying to access the base class, must be overridden"); + } + } + +UrlbarMuxer +~~~~~~~~~~~ + +The *Muxer* is responsible for sorting results based on their importance and +additional rules that depend on the UrlbarQueryContext. The muxer to use is +indicated by the UrlbarQueryContext.muxer property. + +.. caution:: + + The Muxer is a replaceable component, as such what is described here is a + reference for the default View, but may not be valid for other implementations. + +.. highlight:: JavaScript +.. code:: + + class UrlbarMuxer { + /** + * Unique name for the muxer, used by the context to sort results. + * Not using a unique name will cause the newest registration to win. + * @abstract + */ + get name() { + return "UrlbarMuxerBase"; + } + /** + * Sorts UrlbarQueryContext results in-place. + * @param {UrlbarQueryContext} queryContext the context to sort results for. + * @abstract + */ + sort(queryContext) { + throw new Error("Trying to access the base class, must be overridden"); + } + } + + +The Controller +-------------- + +`UrlbarController <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarController.jsm>`_ +is the component responsible for reacting to user's input, by communicating +proper course of action to the Model (e.g. starting/stopping a query) and the +View (e.g. showing/hiding a panel). It is also responsible for reporting Telemetry. + +.. note:: + + Each *View* has a different *Controller* instance. + +.. highlight:: JavaScript +.. code:: + + UrlbarController { + async startQuery(queryContext); + cancelQuery(queryContext); + // Invoked by the ProvidersManager when results are available. + receiveResults(queryContext); + // Used by the View to listen for results. + addQueryListener(listener); + removeQueryListener(listener); + } + + +The View +-------- + +The View is the component responsible for presenting search results to the +user and handling their input. + +.. caution + + The View is a replaceable component, as such what is described here is a + reference for the default View, but may not be valid for other implementations. + +`UrlbarInput.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarInput.jsm>`_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Implements an input box *View*, owns an *UrlbarView*. + +.. highlight:: JavaScript +.. code:: + + UrlbarInput { + constructor(options = { textbox, panel }); + // Uses UrlbarValueFormatter to highlight the base host, search aliases + // and to keep the host visible on overflow. + formatValue(val); + openResults(); + // Converts an internal URI (e.g. a URI with a username or password) into + // one which we can expose to the user. + makeURIReadable(uri); + // Handles an event which would cause a url or text to be opened. + handleCommand(); + // Called by the view when a result is selected. + resultsSelected(); + // The underlying textbox + textbox; + // The results panel. + panel; + // The containing window. + window; + // The containing document. + document; + // An UrlbarController instance. + controller; + // An UrlbarView instance. + view; + // Whether the current value was typed by the user. + valueIsTyped; + // Whether the context is in Private Browsing mode. + isPrivate; + // Whether the input box is focused. + focused; + // The go button element. + goButton; + // The current value, can also be set. + value; + } + +`UrlbarView.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarView.jsm>`_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Represents the base *View* implementation, communicates with the *Controller*. + +.. highlight:: JavaScript +.. code:: + + UrlbarView { + // Manage View visibility. + open(); + close(); + // Invoked when the query starts. + onQueryStarted(queryContext); + // Invoked when new results are available. + onQueryResults(queryContext); + // Invoked when the query has been canceled. + onQueryCancelled(queryContext); + // Invoked when the query is done. This is invoked in any case, even if the + // query was canceled earlier. + onQueryFinished(queryContext); + // Invoked when the view opens. + onViewOpen(); + // Invoked when the view closes. + onViewClose(); + } + + +UrlbarResult +------------ + +An `UrlbarResult <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarResult.jsm>`_ +instance represents a single search result with a result type, that +identifies specific kind of results. +Each kind has its own properties, that the *View* may support, and a few common +properties, supported by all of the results. + +.. note:: + + Result types are also enumerated by *UrlbarUtils.RESULT_TYPE*. + +.. highlight:: JavaScript +.. code:: + + UrlbarResult { + constructor(resultType, payload); + + type: {integer} One of UrlbarUtils.RESULT_TYPE. + source: {integer} One of UrlbarUtils.RESULT_SOURCE. + title: {string} A title that may be used as a label for this result. + icon: {string} Url of an icon for this result. + payload: {object} Object containing properties for the specific RESULT_TYPE. + autofill: {object} An object describing the text that should be + autofilled in the input when the result is selected, if any. + autofill.value: {string} The autofill value. + autofill.selectionStart: {integer} The first index in the autofill + selection. + autofill.selectionEnd: {integer} The last index in the autofill selection. + suggestedIndex: {integer} Suggest a preferred position for this result + within the result set. + } + +The following RESULT_TYPEs are supported: + +.. highlight:: JavaScript +.. code:: + + // An open tab. + // Payload: { icon, url, userContextId } + TAB_SWITCH: 1, + // A search suggestion or engine. + // Payload: { icon, suggestion, keyword, query, providesSearchMode, inPrivateWindow, isPrivateEngine } + SEARCH: 2, + // A common url/title tuple, may be a bookmark with tags. + // Payload: { icon, url, title, tags } + URL: 3, + // A bookmark keyword. + // Payload: { icon, url, keyword, postData } + KEYWORD: 4, + // A WebExtension Omnibox result. + // Payload: { icon, keyword, title, content } + OMNIBOX: 5, + // A tab from another synced device. + // Payload: { icon, url, device, title } + REMOTE_TAB: 6, + // An actionable message to help the user with their query. + // textData and buttonTextData are objects containing an l10n id and args. + // If a tip is untranslated it's possible to provide text and buttonText. + // Payload: { icon, textData, buttonTextData, [buttonUrl], [helpUrl] } + TIP: 7, + // A type of result created at runtime, for example by an extension. + // Payload: { dynamicType } + DYNAMIC: 8, diff --git a/browser/components/urlbar/docs/telemetry.rst b/browser/components/urlbar/docs/telemetry.rst new file mode 100644 index 0000000000..6ebb5f0d07 --- /dev/null +++ b/browser/components/urlbar/docs/telemetry.rst @@ -0,0 +1,445 @@ +Telemetry +========= + +This section describes existing telemetry probes measuring interaction with the +Address Bar. + +.. toctree:: + :caption: Table of Contents + + telemetry + +Histograms +---------- + +PLACES_AUTOCOMPLETE_1ST_RESULT_TIME_MS + This probe tracks the amount of time it takes to get the first result. + It is an exponential histogram with values between 5 and 100. + +PLACES_AUTOCOMPLETE_6_FIRST_RESULTS_TIME_MS + This probe tracks the amount of time it takes to get the first six results. + It is an exponential histogram with values between 50 and 1000. + +FX_URLBAR_SELECTED_RESULT_METHOD + This probe tracks how a result was picked by the user from the list. + It is a categorical histogram with these values: + + - ``enter`` + The user pressed Enter without selecting a result first. + This most likely happens when the user confirms the default preselected + result (aka *heuristic result*), or when they select with the keyboard a + one-off search button and confirm with Enter. + - ``enterSelection`` + The user selected a result, but not using Tab or the arrow keys, and then + pressed Enter. This is a rare and generally unexpected event, there may be + exotic ways to select a result we didn't consider, that are tracked here. + Look at arrowEnterSelection and tabEnterSelection for more common actions. + - ``click`` + The user clicked on a result. + - ``arrowEnterSelection`` + The user selected a result using the arrow keys, and then pressed Enter. + - ``tabEnterSelection`` + The first key the user pressed to select a result was the Tab key, and then + they pressed Enter. Note that this means the user could have used the arrow + keys after first pressing the Tab key. + - ``rightClickEnter`` + Before QuantumBar, it was possible to right-click a result to highlight but + not pick it. Then the user could press Enter. This is no more possible. + +Scalars +------- + +urlbar.tips + This is a keyed scalar whose values are uints and are incremented each time a + tip result is shown, a tip is picked, and a tip's help button is picked. The + keys are: + + - ``intervention_clear-help`` + Incremented when the user picks the help button in the clear-history search + intervention. + - ``intervention_clear-picked`` + Incremented when the user picks the clear-history search intervention. + - ``intervention_clear-shown`` + Incremented when the clear-history search intervention is shown. + - ``intervention_refresh-help`` + Incremented when the user picks the help button in the refresh-Firefox + search intervention. + - ``intervention_refresh-picked`` + Incremented when the user picks the refresh-Firefox search intervention. + - ``intervention_refresh-shown`` + Incremented when the refresh-Firefox search intervention is shown. + - ``intervention_update_ask-help`` + Incremented when the user picks the help button in the update_ask search + intervention, which is shown when there's a Firefox update available but the + user's preference says we should ask them to download and apply it. + - ``intervention_update_ask-picked`` + Incremented when the user picks the update_ask search intervention. + - ``intervention_update_ask-shown`` + Incremented when the update_ask search intervention is shown. + - ``intervention_update_refresh-help`` + Incremented when the user picks the help button in the update_refresh search + intervention, which is shown when the user's browser is up to date but they + triggered the update intervention. We show this special refresh intervention + instead. + - ``intervention_update_refresh-picked`` + Incremented when the user picks the update_refresh search intervention. + - ``intervention_update_refresh-shown`` + Incremented when the update_refresh search intervention is shown. + - ``intervention_update_restart-help`` + Incremented when the user picks the help button in the update_restart search + intervention, which is shown when there's an update and it's been downloaded + and applied. The user needs to restart to finish. + - ``intervention_update_restart-picked`` + Incremented when the user picks the update_restart search intervention. + - ``intervention_update_restart-shown`` + Incremented when the update_restart search intervention is shown. + - ``intervention_update_web-help`` + Incremented when the user picks the help button in the update_web search + intervention, which is shown when we can't update the browser or possibly + even check for updates for some reason, so the user should download the + latest version from the web. + - ``intervention_update_web-picked`` + Incremented when the user picks the update_web search intervention. + - ``intervention_update_web-shown`` + Incremented when the update_web search intervention is shown. + - ``tabtosearch_onboard-shown`` + Incremented when a tab-to-search onboarding result is shown, once per engine + per engagement. Please note that the number of times tab-to-search + onboarding results are picked is the sum of all keys in + ``urlbar.searchmode.tabtosearch_onboard``. + - ``searchTip_onboard-picked`` + Incremented when the user picks the onboarding search tip. + - ``searchTip_onboard-shown`` + Incremented when the onboarding search tip is shown. + - ``searchTip_redirect-picked`` + Incremented when the user picks the redirect search tip. + - ``searchTip_redirect-shown`` + Incremented when the redirect search tip is shown. + +urlbar.searchmode.* + This is a set of keyed scalars whose values are uints incremented each + time search mode is entered in the Urlbar. The suffix on the scalar name + describes how search mode was entered. Possibilities include: + + - ``bookmarkmenu`` + Used when the user selects the Search Bookmarks menu item in the Library + menu. + - ``handoff`` + Used when the user uses the search box on the new tab page and is handed off + to the address bar. + - ``keywordoffer`` + Used when the user selects a keyword offer result. + - ``oneoff`` + Used when the user selects a one-off engine in the Urlbar. + - ``shortcut`` + Used when the user enters search mode with a keyboard shortcut or menu bar + item (e.g. ``Accel+K``). + - ``tabmenu`` + Used when the user selects the Search Tabs menu item in the tab overflow + menu. + - ``tabtosearch`` + Used when the user selects a tab-to-search result. These results suggest a + search engine when the search engine's domain is autofilled. + - ``tabtosearch_onboard`` + Used when the user selects a tab-to-search onboarding result. These are + shown the first few times the user encounters a tab-to-search result. + - ``topsites_newtab`` + Used when the user selects a search shortcut Top Site from the New Tab Page. + - ``topsites_urlbar`` + Used when the user selects a search shortcut Top Site from the Urlbar. + - ``touchbar`` + Used when the user taps a search shortct on the Touch Bar, available on some + Macs. + - ``typed`` + Used when the user types an engine alias in the Urlbar. + - ``other`` + Used as a catchall for other behaviour. We don't expect this scalar to hold + any values. If it does, we need to correct an issue with search mode entry + points. + + The keys for the scalars above are engine and source names. If the user enters + a remote search mode with a built-in engine, we record the engine name. If the + user enters a remote search mode with an engine they installed (e.g. via + OpenSearch or a WebExtension), we record ``other`` (not to be confused with + the ``urlbar.searchmode.other`` scalar above). If they enter a local search + mode, we record the English name of the result source (e.g. "bookmarks", + "history", "tabs"). Note that we slightly modify the engine name for some + built-in engines: we flatten all localized Amazon sites (Amazon.com, + Amazon.ca, Amazon.de, etc.) to "Amazon" and we flatten all localized + Wikipedia sites (Wikipedia (en), Wikipedia (fr), etc.) to "Wikipedia". This + is done to reduce the number of keys used by these scalars. + +urlbar.picked.* + This is a set of keyed scalars whose values are uints incremented each + time a result is picked from the Urlbar. The suffix on the scalar name + is the result type. The keys for the scalars above are the 0-based index of + the result in the urlbar panel when it was picked. + + .. note:: + Available from Firefox 84 on. Use the *FX_URLBAR_SELECTED_** histograms in + earlier versions. See the `Obsolete probes`_ section below. + + Valid result types are: + + - ``autofill`` + An origin or a URL completed the user typed text inline. + - ``bookmark`` + A bookmarked URL. + - ``dynamic`` + A specially crafted result, often used in experiments when basic types are + not flexible enough for a rich layout. + - ``extension`` + Added by an add-on through the omnibox WebExtension API. + - ``formhistory`` + A search suggestion from previous search history. + - ``history`` + A URL from history. + - ``keyword`` + A bookmark keyword. + - ``remotetab`` + A tab synced from another device. + - ``searchengine`` + A search result, but not a suggestion. May be the default search action + or a search alias. + - ``searchsuggestion`` + A remote search suggestion. + - ``switchtab`` + An open tab. + - ``tabtosearch`` + A tab to search result. + - ``tip`` + A tip result. + - ``topsite`` + An entry from top sites. + - ``unknown`` + An unknown result type, a bug should be filed to figure out what it is. + - ``visiturl`` + The user typed string can be directly visited. + +urlbar.picked.searchmode.* + This is a set of keyed scalars whose values are uints incremented each time a + result is picked from the Urlbar while the Urlbar is in search mode. The + suffix on the scalar name is the search mode entry point. The keys for the + scalars are the 0-based index of the result in the urlbar panel when it was + picked. + + .. note:: + These scalars share elements of both ``urlbar.picked.*`` and + ``urlbar.searchmode.*``. Scalar name suffixes are search mode entry points, + like ``urlbar.searchmode.*``. The keys for these scalars are result indices, + like ``urlbar.picked.*``. + + .. note:: + These data are a subset of the data recorded by ``urlbar.picked.*``. For + example, if the user enters search mode by clicking a one-off then selects + a Google search suggestion at index 2, we would record in **both** + ``urlbar.picked.searchsuggestion`` and ``urlbar.picked.searchmode.oneoff``. + + +Event Telemetry +--------------- + +The event telemetry is grouped under the ``urlbar`` category. + +Event Method + There are two methods to describe the interaction with the urlbar: + + - ``engagement`` + It is defined as a completed action in urlbar, where a user inserts text + and executes one of the actions described in the Event Object. + - ``abandonment`` + It is defined as an action where the user inserts text but does not + complete an engagement action, usually unfocusing the urlbar. This also + happens when the user switches to another window, regardless of urlbar + focus. + +Event Value + This is how the user interaction started + + - ``typed``: The text was typed into the urlbar. + - ``dropped``: The text was drag and dropped into the urlbar. + - ``pasted``: The text was pasted into the urlbar. + - ``topsites``: The user opened the urlbar view without typing, dropping, + or pasting. + In these cases, if the urlbar input is showing the URL of the loaded page + and the user has not modified the input’s content, the urlbar views shows + the user’s top sites. Otherwise, if the user had modified the input’s + content, the urlbar view shows results based on what the user has typed. + To tell whether top sites were shown, it's enough to check whether value is + ``topsites``. To know whether the user actually picked a top site, check + check that ``numChars`` == 0. If ``numChars`` > 0, the user initially opened + top sites, but then they started typing and confirmed a different result. + - ``returned``: The user abandoned a search, for example by switching to + another tab/window, or focusing something else, then came back to it + and continued. We consider a search continued if the user kept at least the + first char of the original search string. + - ``restarted``: The user abandoned a search, for example by switching to + another tab/window, or focusing something else, then came back to it, + cleared it and then typed a new string. + +Event Object + These describe actions in the urlbar: + + - ``click`` + The user clicked on a result. + - ``enter`` + The user confirmed a result with Enter. + - ``drop_go`` + The user dropped text on the input field. + - ``paste_go`` + The user used Paste & Go feature. It is not the same as paste and Enter. + - ``blur`` + The user unfocused the urlbar. This is only valid for ``abandonment``. + +Event Extra + This object contains additional information about the interaction. + Extra is a key-value store, where all the keys and values are strings. + + - ``elapsed`` + Time in milliseconds from the initial interaction to an action. + - ``numChars`` + Number of input characters the user typed or pasted at the time of + submission. + - ``numWords`` + Number of words in the input. The measurement is taken from a trimmed input + split up by its spaces. This is not a perfect measurement, since it will + return an incorrect value for languages that do not use spaces or URLs + containing spaces in its query parameters, for example. + - ``selType`` + The type of the selected result at the time of submission. + This is only present for ``engagement`` events. + It can be one of: ``none``, ``autofill``, ``visiturl``, ``bookmark``, + ``history``, ``keyword``, ``searchengine``, ``searchsuggestion``, + ``switchtab``, ``remotetab``, ``extension``, ``oneoff``, ``keywordoffer``, + ``canonized``, ``tip``, ``tiphelp``, ``formhistory``, ``tabtosearch``, + ``unknown`` + In practice, ``tabtosearch`` should not appear in real event telemetry. + Opening a tab-to-search result enters search mode and entering search mode + does not currently mark the end of an engagement. It is noted here for + completeness. + - ``selIndex`` + Index of the selected result in the urlbar panel, or -1 for no selection. + There won't be a selection when a one-off button is the only selection, and + for the ``paste_go`` or ``drop_go`` objects. There may also not be a + selection if the system was busy and results arrived too late, then we + directly decide whether to search or visit the given string without having + a fully built result. + This is only present for ``engagement`` events. + - ``provider`` + The name of the result provider for the selected result. Existing values + are: ``HeuristicFallback``, ``Autofill``, ``UnifiedComplete``, + ``TokenAliasEngines``, ``SearchSuggestions``, ``UrlbarProviderTopSites``. + Values can also be defined by `URLBar provider experiments`_. + + .. _URLBar provider experiments: experiments.html#developing-address-bar-extensions + +Search probes relevant to the Address Bar +----------------------------------------- + +SEARCH_COUNTS + This histogram tracks search engines and Search Access Points. It is augmented + by multiple SAPs, including the urlbar. + It's a keyed histogram, the keys are strings made up of search engine names + and SAP names, for example ``google.urlbar``. + For each key, this records the count of searches made using that engine and SAP. + SAP names can be: + + - ``alias`` This is when using an alias (like ``@google``) in the urlbar. + Note there is often confusion between the terms alias and keyword, and + they may be used inappropriately: aliases refer to search engines, while + keywords refer to bookmarks. We expect no results for this SAP in Firefox + 83+, since urlbar-searchmode replaces it. + - ``abouthome`` + - ``contextmenu`` + - ``newtab`` + - ``searchbar`` + - ``system`` + - ``urlbar`` Except aliases and search mode. + - ``urlbar-searchmode`` Used when the Urlbar is in search mode. + - ``webextension`` + - ``oneoff-urlbar`` + - ``oneoff-searchbar`` + - ``unknown`` This is actually the searchbar, when using the current engine + one-off button. + +browser.engagement.navigation.* + These keyed scalars track search through different SAPs, for example the + urlbar is tracked by ``browser.engagement.navigation.urlbar``. + It counts loads triggered in a subsession from the specified SAP, broken down + by the originating action. + Possible SAPs are: + + - ``urlbar`` Except search mode. + - ``urlbar_searchmode`` Used when the Urlbar is in search mode. + - ``searchbar`` + - ``about_home`` + - ``about_newtab`` + - ``contextmenu`` + - ``webextension`` + - ``system`` Indicates a search from the command line. + + Recorded actions may be: + + - ``search`` + Used for any search from ``contextmenu``, ``system`` and ``webextension``. + - ``search_alias`` + For ``urlbar``, indicates the user confirmed a search through an alias. + - ``search_enter`` + For ``about_home`` and ``about:newtab`` this counts any search. + For the other SAPs it tracks typing and then pressing Enter. + - ``search_formhistory`` + For ``urlbar``, indicates the user picked a form history result. + - ``search_oneoff`` + For ``urlbar`` or ``searchbar``, indicates the user confirmed a search + using a one-off button. + - ``search_suggestion`` + For ``urlbar`` or ``searchbar``, indicates the user confirmed a search + suggestion. + +Obsolete probes +--------------- + +Obsolete histograms +~~~~~~~~~~~~~~~~~~~ + +FX_URLBAR_SELECTED_RESULT_INDEX (OBSOLETE) + This probe tracked the indexes of picked results in the results list. + It was an enumerated histogram with 17 buckets. + +FX_URLBAR_SELECTED_RESULT_TYPE and FX_URLBAR_SELECTED_RESULT_TYPE_2 (from Firefox 78 on) (OBSOLETE) + This probe tracked the types of picked results. + It was an enumerated histogram with 17 buckets: + + 0. autofill + 1. bookmark + 2. history + 3. keyword + 4. searchengine + 5. searchsuggestion + 6. switchtab + 7. tag + 8. visiturl + 9. remotetab + 10. extension + 11. preloaded-top-site + 12. tip + 13. topsite + 14. formhistory + 15. dynamic + 16. tabtosearch + +FX_URLBAR_SELECTED_RESULT_INDEX_BY_TYPE and FX_URLBAR_SELECTED_RESULT_INDEX_BY_TYPE_2 (from Firefox 78 on) (OBSOLETE) + This probe tracked picked result type, for each one it tracked the index where + it appeared. + It was a keyed histogram where the keys were result types (see + FX_URLBAR_SELECTED_RESULT_TYPE above). For each key, this recorded the indexes + of picked results for that result type. + +Obsolete search probes +---------------------- + +navigation.search (OBSOLETE) + This is a legacy and disabled event telemetry that is currently under + discussion for removal or modernization. It can't be enabled through a pref. + it's more or less equivalent to browser.engagement.navigation, but can also + report the picked search engine. diff --git a/browser/components/urlbar/docs/utilities.rst b/browser/components/urlbar/docs/utilities.rst new file mode 100644 index 0000000000..8fa3b8509e --- /dev/null +++ b/browser/components/urlbar/docs/utilities.rst @@ -0,0 +1,26 @@ +Utilities +========= + +Various modules provide shared utilities to the other components: + +`UrlbarPrefs.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarPrefs.jsm>`_ +------------------------------------------------------------------------------------------------------------- + +Implements a Map-like storage or urlbar related preferences. The values are kept +up-to-date. + +.. highlight:: JavaScript +.. code:: + + // Always use browser.urlbar. relative branch, except for the preferences in + // PREF_OTHER_DEFAULTS. + UrlbarPrefs.get("delay"); // Gets value of browser.urlbar.delay. + +.. note:: + + Newly added preferences should always be properly documented in UrlbarPrefs. + +`UrlbarUtils.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarUtils.jsm>`_ +------------------------------------------------------------------------------------------------------------- + +Includes shared utils and constants shared across all the components. |