diff options
Diffstat (limited to 'toolkit/components/glean/docs')
| -rw-r--r-- | toolkit/components/glean/docs/api.md | 4 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/code_organization.md | 64 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/images/fog-modules.svg | 6 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/index.md | 32 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/ipc.md | 106 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/new_definitions_file.md | 85 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/new_metric_types.md | 114 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/preferences.md | 21 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/storage.md | 14 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/style_guide.md | 58 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/testing.md | 117 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/updating_parser.md | 37 | ||||
| -rw-r--r-- | toolkit/components/glean/docs/updating_sdk.md | 39 |
13 files changed, 697 insertions, 0 deletions
diff --git a/toolkit/components/glean/docs/api.md b/toolkit/components/glean/docs/api.md new file mode 100644 index 0000000000..8561d11d3b --- /dev/null +++ b/toolkit/components/glean/docs/api.md @@ -0,0 +1,4 @@ +# API + +The Metrics API documentation can be found in the +[Glean SDK Book](https://mozilla.github.io/glean/book/user/metrics/index.html). diff --git a/toolkit/components/glean/docs/code_organization.md b/toolkit/components/glean/docs/code_organization.md new file mode 100644 index 0000000000..2055fd0469 --- /dev/null +++ b/toolkit/components/glean/docs/code_organization.md @@ -0,0 +1,64 @@ +# FOG code organization + +```eval_rst +.. note:: + + Project FOG is currently being designed and implemented. + This documentation is incomplete and may change significantly before FOG is usable inside mozilla-central. +``` + + + +The diagram shows the different modules of Project FOG. + +## FOG control + +This module is the glue between Firefox and Glean. + +* The code lives in `toolkit/components/glean/src`. +* It is written in Rust. +* The crate is named `fog_control`. +* It is not published to crates.io. +* It is not consumed by other Rust crates inside mozilla-central. + +This module is responsible for + +* collecting and assembling the [client information](https://mozilla.github.io/glean/book/user/pings/index.html#the-client_info-section) +* configuring Glean +* watching the Firefox Telemetry data upload preference (`datareporting.healthreport.uploadEnabled`) +* scheduling builtin pings +* controling ping upload workers +* passing IPC buffers + +It calls into `glean_core` to: + +* configure and initialize Glean +* toggle `upload_enabled` +* get upload tasks + +It calls into `fog` to: + +* pass IPC buffers +* record to its own metrics + +## FOG API + +This module provides the user-facing API for Glean inside mozilla-central. + +* The code lives in `toolkit/components/glean/api`. +* It is written in Rust. +* The crate is named `fog`. +* It is not published to crates.io. +* It can be consumed by other Rust crates inside mozilla-central for their Glean usage. +* It will provide a build task for `glean_parser` integration. + +This module is responsible for + +* exposing a specific metric API in Rust +* wrapping metric implementations for handling IPC +* exposing FFI functionality to implement other language APIs on top + +It calls into `glean_core` for: + +* metric types (including pings) +* querying `upload_enabled` status. diff --git a/toolkit/components/glean/docs/images/fog-modules.svg b/toolkit/components/glean/docs/images/fog-modules.svg new file mode 100644 index 0000000000..2b188c67a1 --- /dev/null +++ b/toolkit/components/glean/docs/images/fog-modules.svg @@ -0,0 +1,6 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- This Source Code Form is subject to the terms of the Mozilla Public + - License, v. 2.0. If a copy of the MPL was not distributed with this + - file, You can obtain one at http://mozilla.org/MPL/2.0/. --> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg xmlns="http://www.w3.org/2000/svg" style="background-color: rgb(255, 255, 255);" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="601px" height="511px" viewBox="-0.5 -0.5 601 511" content="<mxfile host="app.diagrams.net" modified="2020-03-27T09:36:49.445Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:76.0) Gecko/20100101 Firefox/76.0" etag="_te5ruI7nKv5hlBG0WM7" version="12.9.3" type="device"><diagram id="02tFoor6QwIjLFNeosYU" name="Page-1">7Vtbd6M2EP41fkyPQYDxo+NcmnbbpOvu2eZRBtmmxYgVsmP311eAZC6SDU4AX5ok5wSNLsA334xGI9ED4+XmkcBw8Rt2kd/T++6mB+56uq4NLYP9iyVbLtHsYSqZE8/lskww8f5FXNjn0pXnoqjQkGLsUy8sCh0cBMihBRkkBL8Vm82wX7xrCOdIEkwc6MvS755LF6nUNvuZ/GfkzRfizlqf1yyhaMwF0QK6+C0nAvc9MCYY0/RquRkjP0ZP4JL2e9hTu3swggJap0P/YUNGP0L/5QvY+N8XzwT/Y93wUdbQX/EX5g9LtwIBgleBi+JB+j1w+7bwKJqE0Ilr35jSmWxBlz4raexy5vn+GPuYJH3BbDbTHYfJI8puh3I1rjW1TIvVyK8hngkRijY5EX+tR4SXiJItayJqBcScZBYvvmUKs4RaFnlliX6Qk2S+GzrDkV1wKI+AVW8XVtdEtmuoYLX1KbBaglWza+IKQFu4AgWuls9ue+t66/iGvjcPkgrrxyo2rFuSPNauyK7m/H/SbYYZPHm9iIZxxU2UuKMRa6BZ4UYe5Q6tkY9DRGLPAh0vmItx2dulQxdvx8TJgwppiRNMMbSo+KKCAxygEhu4SLx4+rrgNtayx7zYiMuXnuvGd1ESrUjFBphj2pUGqRkq4rTFG6OCN0frv2umPQUUkQD60SfBEoINq12TBrpkmGZKYCOXRRK8iAld4DlmGrzPpCVcsjZfMA65fv5GlG55WARXFBe1hzYe/Svu/pPJS6+5mrsNHzkpbEUhYO+b6xQXX/N1WbektK1SW4RXxEEHsLF43AbJHPGuT9/MP23zj8lo8818CbSvweDGFsFIjNtBEhDkQ+qtixGaSqNJ1xEhcJtrEGIvoFFu5JdYkHELDErcGpbiqlJ73TjYnl2kT5Bxa/cq76ebJTk0Ns5t8tdk+GY7SB2+TW3TMBsyZqAVARwqbFlX2LLVli0PJXB/gWs4cYgX0qNdah6sffAK5+qg2M2f3Lsa9ukUonQMxqdv3Tvv6BflXM3DzvI8nKvAtJ4DOMrBXrh1quGyPs1zLzYiZ5UzT3VDcBb2qfdL9jaosM/D7VuyT1XWKrecSxc/X1cRrbks+lAuBiJ7poyRLMdG01lDuRjj3IxeTsU8PD8ywejlieEMlzFqwTQK08Vpqdy/AhXs1pInC4tqZBk7cMTvd44CrErnqDcevHwsu9vfw/0xZtMG9q+A3IZxagejy07+0UcwYKLJ3a8XCLE+PDuIZQeSTpoXh+1uG6IKy/Z2fOT58HIzMlYpqqs707WHrimh+/Qyzr3mg4zyAi+nq6iTJVk5g7UDIgeYrcDLbg2vQY3QIHBH8QY5Kzk+jCLPKS2w3rlYes/C7AMxhNl0DJHTmanQmZAdtw6TFk5GycTMspdP35z3OrACM8zSQFppoBQZaaCmFmO63STVCkTLeFdBNS1HtIx2TVNNLHmqqWZ+Uq0VqsmJ+df7icS2j29ynkkeHpTw3k0zOd6ottFbm4ZFHJBTwO/PV4x/KWLXFRF7t/i3fVzLjH+VwXry00tPK+Tk6U87iW1V2KmrNvV3ScjmAa+TYmk7juogihKoVk5toG6a+n8ytR29/1V2KVYH+XJwmg2azjdbQN0ADRg1WdxNPhHIuQNKYBDNkoOFKHCwmxwtvLi0jAlKbB/IHr3bRAKwT2IKF7FXKayi2nzqrm86Mh/5dGeaHIrY1NxsfNRJ9q1sNaqwv1urMcCn1VQZQ7XVDM/LavalVAly1tdgNapD591aTY3V2lnvzzbP2D3Bdyl9bgy6Cb6NisMt+56r3WBdTnLlgkEXXU0waJx8z86QV/epC/ThNkb7WvNaVo2TLIOWFHHobKriPFftr2Y01VczsS7T80jqb2YubpLbqUBorr2MMCtm36amzi37xBfc/wc=</diagram></mxfile>"><defs/><g><rect x="0" y="60" width="600" height="120" fill="#fff2cc" stroke="#d6b656" pointer-events="all"/><rect x="0" y="180" width="600" height="330" fill="#d5e8d4" stroke="#82b366" pointer-events="all"/><rect x="460" y="60" width="140" height="30" fill="none" stroke="none" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe flex-end; width: 138px; height: 1px; padding-top: 75px; margin-left: 460px;"><div style="box-sizing: border-box; font-size: 0; text-align: right; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; "><div align="right"><font style="font-size: 16px">Developer facing</font></div></div></div></div></foreignObject><text x="598" y="79" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="end">Developer facing</text></switch></g><rect x="470" y="180" width="130" height="30" fill="none" stroke="none" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe flex-end; width: 128px; height: 1px; padding-top: 195px; margin-left: 470px;"><div style="box-sizing: border-box; font-size: 0; text-align: right; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; "><div style="font-size: 16px" align="right"><font style="font-size: 16px">Internals</font></div></div></div></div></foreignObject><text x="598" y="199" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="end">Internals</text></switch></g><path d="M 250 150 L 250 190 L 120 190 L 120 203.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 120 208.88 L 116.5 201.88 L 120 203.63 L 123.5 201.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="190" y="90" width="120" height="60" fill="#f8cecc" stroke="#b85450" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 120px; margin-left: 191px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">C++</div></div></div></foreignObject><text x="250" y="124" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">C++</text></switch></g><rect x="360" y="90" width="120" height="60" fill="#f8cecc" stroke="#b85450" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 120px; margin-left: 361px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">JavaScript</div></div></div></foreignObject><text x="420" y="124" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">JavaScript</text></switch></g><path d="M 420 150 L 420 190 L 120 190 L 120 203.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 120 208.88 L 116.5 201.88 L 120 203.63 L 123.5 201.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="360" y="90" width="120" height="60" fill="#ffffff" stroke="#000000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 120px; margin-left: 361px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">JavaScript</div></div></div></foreignObject><text x="420" y="124" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">JavaScript</text></switch></g><path d="M 80 150 L 80 170 L 80 223.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 80 228.88 L 76.5 221.88 L 80 223.63 L 83.5 221.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="20" y="90" width="120" height="60" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 120px; margin-left: 21px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; "><div>Rust</div></div></div></div></foreignObject><text x="80" y="124" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">Rust</text></switch></g><rect x="20" y="230" width="120" height="60" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 260px; margin-left: 21px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">FOG API </div></div></div></foreignObject><text x="80" y="264" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">FOG API </text></switch></g><path d="M 140 470 L 160 470 L 150 470 L 163.63 470" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 168.88 470 L 161.88 473.5 L 163.63 470 L 161.88 466.5 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="20" y="440" width="120" height="60" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 470px; margin-left: 21px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">FOG Control</div></div></div></foreignObject><text x="80" y="474" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">FOG Control</text></switch></g><rect x="170" y="440" width="120" height="60" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 470px; margin-left: 171px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">Glean SDK</div></div></div></foreignObject><text x="230" y="474" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">Glean SDK</text></switch></g><rect x="480" y="0" width="120" height="20" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 10px; margin-left: 481px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">Rust</div></div></div></foreignObject><text x="540" y="14" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">Rust</text></switch></g><rect x="480" y="30" width="120" height="20" fill="#f8cecc" stroke="#b85450" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 40px; margin-left: 481px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">C++</div></div></div></foreignObject><text x="540" y="44" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">C++</text></switch></g><path d="M 230 220 L 270 260 L 230 300 L 190 260 Z" fill="#ffffff" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 78px; height: 1px; padding-top: 260px; margin-left: 191px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">IPC parent?</div></div></div></foreignObject><text x="230" y="264" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">IPC parent?</text></switch></g><path d="M 230 300 L 230 433.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 230 438.88 L 226.5 431.88 L 230 433.63 L 233.5 431.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><path d="M 140 260 L 183.63 260" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 188.88 260 L 181.88 263.5 L 183.63 260 L 181.88 256.5 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="230" y="310" width="40" height="20" fill="none" stroke="none" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 38px; height: 1px; padding-top: 320px; margin-left: 231px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">YES</div></div></div></foreignObject><text x="250" y="324" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">YES</text></switch></g><rect x="270" y="240" width="40" height="20" fill="none" stroke="none" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 38px; height: 1px; padding-top: 250px; margin-left: 271px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">NO</div></div></div></foreignObject><text x="290" y="254" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">NO</text></switch></g><rect x="360" y="230" width="230" height="200" fill="#f5f5f5" stroke="#666666" pointer-events="all"/><path d="M 270 260 L 460 260 Q 470 260 470 261.82 L 470 263.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 470 268.88 L 466.5 261.88 L 470 263.63 L 473.5 261.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><path d="M 470 290 L 470 310 L 470 290 L 470 303.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 470 308.88 L 466.5 301.88 L 470 303.63 L 473.5 301.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="410" y="270" width="120" height="20" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 280px; margin-left: 411px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">transfer encoding</div></div></div></foreignObject><text x="470" y="284" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">transfer encoding</text></switch></g><path d="M 470 330 L 470 353.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 470 358.88 L 466.5 351.88 L 470 353.63 L 473.5 351.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="410" y="310" width="120" height="20" fill="#f8cecc" stroke="#b85450" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 320px; margin-left: 411px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">IPC send</div></div></div></foreignObject><text x="470" y="324" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">IPC send</text></switch></g><path d="M 470 380 L 470 393.63" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 470 398.88 L 466.5 391.88 L 470 393.63 L 473.5 391.88 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="410" y="360" width="120" height="20" fill="#f8cecc" stroke="#b85450" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 370px; margin-left: 411px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">IPC recv</div></div></div></foreignObject><text x="470" y="374" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">IPC recv</text></switch></g><path d="M 470 420 L 470 470 L 296.37 470" fill="none" stroke="#000000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 291.12 470 L 298.12 466.5 L 296.37 470 L 298.12 473.5 Z" fill="#000000" stroke="#000000" stroke-miterlimit="10" pointer-events="all"/><rect x="410" y="400" width="120" height="20" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 410px; margin-left: 411px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">transfer decoding</div></div></div></foreignObject><text x="470" y="414" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">transfer decoding</text></switch></g><rect x="520" y="230" width="70" height="20" fill="none" stroke="none" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 68px; height: 1px; padding-top: 240px; margin-left: 521px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; ">IPC layer</div></div></div></foreignObject><text x="555" y="244" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">IPC layer</text></switch></g><rect x="100" y="210" width="40" height="20" fill="#f8cecc" stroke="#b85450" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 38px; height: 1px; padding-top: 220px; margin-left: 101px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: #000000; line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; "><font style="font-size: 11px">C API</font></div></div></div></foreignObject><text x="120" y="224" fill="#000000" font-family="Helvetica" font-size="12px" text-anchor="middle">C API</text></switch></g></g><switch><g requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"/><a transform="translate(0,-5)" xlink:href="https://desk.draw.io/support/solutions/articles/16000042487" target="_blank"><text text-anchor="middle" font-size="10px" x="50%" y="100%">Viewer does not support full SVG 1.1</text></a></switch></svg>
\ No newline at end of file diff --git a/toolkit/components/glean/docs/index.md b/toolkit/components/glean/docs/index.md new file mode 100644 index 0000000000..ad43fa6fee --- /dev/null +++ b/toolkit/components/glean/docs/index.md @@ -0,0 +1,32 @@ +# Firefox on Glean (FOG) + +Firefox on Glean (FOG) is the name of the layer that integrates the +[Glean SDK][glean-sdk] into +[Firefox Desktop](https://www.firefox.com/). +It is currently being designed and implemented. + +The [Glean SDK][glean-sdk] +is a data collection library built by Mozilla for use in its products. +Like [Telemetry][telemetry], it can be used to +(in accordance with our [Privacy Policy][privacy-policy]) +send anonymous usage statistics to Mozilla in order to make better decisions. + +If you have any questions, please reach out to the team on +[#glean:mozilla.org][glean-matrix]. + +```eval_rst +.. toctree:: + :titlesonly: + :maxdepth: 1 + :glob: + + Glean SDK Source <https://github.com/mozilla/glean/> + Glean SDK Documentation <https://mozilla.github.io/glean/book/index.html> + * + +``` +[telemetry]: ../telemetry +[glean-sdk]: https://github.com/mozilla/glean/ +[book-of-glean]: https://mozilla.github.io/glean/book/index.html +[privacy-policy]: https://www.mozilla.org/privacy/ +[glean-matrix]: https://chat.mozilla.org/#/room/#glean:mozilla.org diff --git a/toolkit/components/glean/docs/ipc.md b/toolkit/components/glean/docs/ipc.md new file mode 100644 index 0000000000..2b17b6b982 --- /dev/null +++ b/toolkit/components/glean/docs/ipc.md @@ -0,0 +1,106 @@ +# Inter-process Communication (IPC) + +Firefox Desktop is a multi-process desktop application. +Code requiring instrumentation may be on any of its processes, +so FOG needs to provide facilities to enable that. + +## Design + +The IPC Design of FOG was worked out in +[bug 1618253](https://bugzilla.mozilla.org/show_bug.cgi?id=1618253). + +It centred around a few specific concepts: + +### Forbidding Non-Commutative Operations + +Because we cannot nicely impose a canonical ordering of metric operations across all processes, +FOG forbids non-[commutative](https://en.wikipedia.org/wiki/Commutative_property) +metric operations in some circumstances. + +For example, +`Add()`-ing to a Counter metric works from multiple processes because the order doesn't matter. +However, given a String metric being `Set()` from multiple processes simultaneously, +which value should it take? + +This ambiguity is not a good foundation to build trust on, +so we forbid setting a String metric from multiple processes. + +#### List of Forbidden Operations + +* Boolean's `set` (this is the metric type's only operation) +* Labeled Boolean's `set` (this is the metric type's only operation) +* String's `set` (this is the metric type's only operation) +* Labeled String's `set` (this is the metric type's only operation) +* String List's `set` + * `add` is permitted (order and uniqueness are not guaranteed) +* Timespan's `start`, `stop`, and `cancel` (these are the metric type's only operations) +* UUID's `set` and `generateAndSet` (these are the metric type's only operations) +* Datetime's `set` (this is the metric type's only operation) +* Quantity's `set` (this is the metric type's only operation) + +This list may grow over time as new metric types are added. +If there's an operation/metric type on this list that you need to use in a non-parent process, +please reach out +[on the #glean channel](https://chat.mozilla.org/#/room/#glean:mozilla.org) +and we'll help you out. + +### Process Agnosticism + +For metric types that can be used cross-process, +FOG provides no facility for identifying which process the instrumentation is on. + +What this means is that if you accumulate to a +[Timing Distribution](https://mozilla.github.io/glean/book/user/metrics/timing_distribution.html) +in multiple processes, +all the samples from all the processes will be combined in the same metric. + +If you wish to distinguish samples from different process types, +you will need multiple metrics and inline code to select the proper one for the given process. +For example: + +```C++ +if (XRE_GetProcessType() == GeckoProcessType_Default) { + mozilla::glean::performance::cache_size.Accumulate(numBytes / 1024); +} else { + mozilla::glean::performance::non_main_process_cache_size.Accumulate(numBytes / 1024); +} +``` + +### Scheduling + +FOG makes no guarantee about when non-main-process metric values are sent across IPC. +FOG will try its best to schedule opportunistically in idle moments. + +There are a few cases where we provide more firm guarantees: + +#### Tests + +There are test-only APIs in Rust, C++, +and Javascript that expose when a complete flush of child process metric values has completed. +See [the test documentation](testing.md) for more details. + +#### Built-in Pings + +[Built-in pings](https://mozilla.github.io/glean/book/user/pings/index.html) +will send only after all metric values from all child processes have been collected. + +We cannot at this time provide the same guarantee for +[Custom Pings](https://mozilla.github.io/glean/book/user/pings/custom.html). + +#### Shutdown + +We will make a best effort during an orderly shutdown to flush all pending data in child processes. + +### Mechanics + +At present +(see [bug 1641989](https://bugzilla.mozilla.org/show_bug.cgi?id=1641989)) +FOG uses messages on the PContent protocol. +This enables communication between content child processes and the parent process. + +The rough design is that the Parent can request an immediate flush of pending data, +and each Child can decide to flush its pending data whenever it wishes. + +Pending Data is a buffer of bytes generated by `bincode` in Rust in the Child, +handed off to C++, passed over IPC, +then given back to `bincode` in Rust on the Parent. diff --git a/toolkit/components/glean/docs/new_definitions_file.md b/toolkit/components/glean/docs/new_definitions_file.md new file mode 100644 index 0000000000..1e5b188caa --- /dev/null +++ b/toolkit/components/glean/docs/new_definitions_file.md @@ -0,0 +1,85 @@ +# New Metrics and Pings + +**Note:** FOG is not ready to be used, +so you probably should not be adding metrics and pings using this guide yet. +Instead, please use Firefox Telemetry unless you have explicitly been given permission by a +[Telemetry Module Peer](https://wiki.mozilla.org/Modules/All#Telemetry). + +To add a new metric or ping to Firefox Desktop you should follow the +[Glean SDK documentation on the subject](https://mozilla.github.io/glean/book/user/adding-new-metrics.html), +with some few twists. + +## IPC + +Firefox Desktop is made of multiple processes. +You can record data from any process in Firefox Desktop +[subject to certain conditions](ipc.md). + +If you will be recording data to this metric in multiple processes, +you should make yourself aware of those conditions. + +## Where do I Define new Metrics and Pings? + +Metrics and pings are defined in their definitions files +(`metrics.yaml` or `pings.yaml`, respectively). +But where can you find `metrics.yaml` or `pings.yaml`? + +If you're not the first person in your component to ask that question, +the answer is likely "in the root of your component". +Look for the definitions files near to where you are instrumenting your code. +Or you can look in +`toolkit/components/glean/metrics_index.py` +to see the list of all currently-known definitions files. + +If you _are_ the first person in your component to ask that question, +you get to choose where to start them! +We recommend adding them in the root of your component, next to a `moz.build`. + +When you do so, be sure to edit `toolkit/components/glean/metrics_index.py`, +adding your definitions files to the Python lists therein. +If you don't, no API will be generated for your metrics and your build will fail. + +In addition, do not forget to file a bug in `Data Platform and Tools :: General` +asking for your definitions files to be added to the others for `firefox.desktop`. +If you don't, your metrics will not show up in datasets and tools +because the pipeline won't know that they exist. + +If you have any questions, be sure to ask on +[the #glean channel](https://chat.mozilla.org/#/room/#glean:mozilla.org). + +**Note:** Do _not_ use `toolkit/components/glean/metrics.yaml` +or `toolkit/components/glean/pings.yaml`. +These are for metrics instrumenting the code under `toolkit/components/glean` +and are not general-purpose locations for adding metrics and pings. + +## How does Expiry Work? + +In FOG, +unlike in other Glean-SDK-using projects, +metrics expire based on Firefox application version. +This is to allow metrics to be valid over the entire life of an application version, +whether that is the 4-6 weeks of usual releases or the 13 months of ESR releases. + +There are three values accepted in the `expires` field of `metrics.yaml`s for FOG: +* `"X"` (where `X` is the major portion of a Firefox Desktop version) - + The metric will be expired when the `MOZ_APP_VERSION` reaches or exceeds `X`. + (For example, when the Firefox Version is `88.0a1`, + all metrics marked with `expires: "88"` or lower will be expired.) + This is the recommended form for all new metrics to ensure they stop recording when they stop being relevant. +* `expired` - For marking a metric as manually expired. + Not usually used, but sometimes helpful for internal tests. +* `never` - For marking a metric as part of a permanent data collection. + Metrics marked with `never` must have [instrumentation tests](testing.md). + +For more information on what expiry means and the +`metrics.yaml` format, see +[the Glean SDK docs](https://mozilla.github.io/glean/book/user/metric-parameters.html) +on this subject. Some quick facts: + +* Data collected to expired metrics is not recorded or sent. +* Recording to expired metrics is not an error at runtime. +* Expired metrics being in a `metrics.yaml` is a linting error in `glean_parser`. +* Expired (and non-expired) metrics that are no longer useful should be promptly removed from your `metrics.yaml`. + This reduces the size and improves the performance of Firefox + (and speeds up the Firefox build process) + by decreasing the amount of code that needs to be generated. diff --git a/toolkit/components/glean/docs/new_metric_types.md b/toolkit/components/glean/docs/new_metric_types.md new file mode 100644 index 0000000000..c950978f95 --- /dev/null +++ b/toolkit/components/glean/docs/new_metric_types.md @@ -0,0 +1,114 @@ +# Adding a New Metric Type + +This document covers how to add a new metric type to FOG. +You should only have to do this if a new metric type is added to the +[Glean SDK](https://mozilla.github.io/glean/book/user/metrics/index.html) +and it is needed in Firefox Desktop. + +## IPC + +For detailed information about the IPC design, +including a list of forbidden operations, +please consult +[the FOG IPC documentation](ipc.md). + +When adding a new metric type, the main IPC considerations are: +* Which operations are forbidden because they are not commutative? + * Most `set`-style operations cannot be reconciled sensibly across multiple processes. +* If there are non-forbidden operations, +what partial representation will this metric have in non-main processes? +Put another way, what shape of storage will this take up in the +[IPC Payload](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/api/src/ipc.rs)? + * For example, Counters can aggregate all partial counts together to a single + "partial sum". So + [its representation](https://searchfox.org/mozilla-central/rev/803b368879fa332e8e2c1840bf1ec164f7ed2c32/toolkit/components/glean/api/src/ipc.rs#45) + in the IPC Payload is just a single number per Counter. + * In contrast, Timing Distributions' durations are calculated in the core, + so it can't calculate samples in child processes. + Instead we will need to record the start and stop instants as pairs, + more or less sending a command stream with timing information. + +To implement IPC support in a metric type, +we split the metric into three pieces: +1. An umbrella `enum` with the name `MetricTypeMetric`. + * It has a `Child` and a `Parent` variant. + * It is IPC-aware and is responsible for + * If on a non-parent-process, + either storing partial representations in the IPC Payload, + or logging errors if forbidden non-test APIs are called. + (Or panicking if test APIs are called.) + * If on the parent process, dispatching API calls to the core. +2. The parent-process implementation is supplied by + [the RLB](https://crates.io/crates/glean/). + * For testing, it stores the `MetricId` that identifies this particular metric in a cross-process fashion. + * For testing, it exposes a `child_metric()` function to create its `Child` equivalent. + * For testing and if it supports operations in a non-parent-process, it exposes a `metric_id()` function to access the stored `MetricId`. +3. The `MetricTypeIpc` is the ipc-aware non-parent-process implementation. + * If it does support operations in non-parent processes it stores the `MetricId` that identifies this particular metric in a cross-process fashion. + +## Rust + +FOG uses the Rust Language Binding APIs (the `glean` crate) with a layer of IPC on top. + +The IPC additions and glean-core trait implementations are in the +[`private` module of the `fog` crate](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/api/src/metrics). + +Each metric type gets its own file, mimicking the structure in +[`glean_core`](https://github.com/mozilla/glean/tree/main/glean-core/src/metrics) +and [`glean`](https://github.com/mozilla/glean/tree/main/glean-core/rlb/src/private). + +Every method on the metric type is public for now, +including test methods. + +## C++ and JS + +The C++ and JS APIs are implemented [atop the Rust API](code_organization.md). +We treat them both together since, though they're different languages, +they're both implemented in C++ and share much of their implementation. + +Each metric type has six pieces you'll need to cover: + +### 1. MLA FFI + +- Using our convenient macros, define the Multi-Language Architecture's FFI layer above the Rust API in [`api/src/ffi/mod.rs`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/api/src/ffi/mod.rs). + +### 2. C++ Impl + +- Implement a type called `XMetric` (e.g. `CounterMetric`) in `mozilla::glean::impl` in [`bindings/private/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/bindings/private/). + - Its methods should be named the same as the ones in the Rust API, transformed to `CamelCase`. + - They should all be public. + - Multiplex the FFI's `test_have` and `test_get` functions into a single `TestGetValue` function that returns a `mozilla::Maybe` wrapping the C++ type that best fits the metric type. +- Include the new metric type in + [`bindings/MetricTypes.h`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/bindings/MetricTypes.h) + and + [`build_scripts/glean_parser_ext/util.py`'s `IMPLEMENTED_CPP_TYPES`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/build_scripts/glean_parser_ext/util.py). +- Include the new files in [`moz.build`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/bindings/MetricTypes.h). The header file should be added to `EXPORTS.mozilla.glean.bindings` and the `.cpp` file should be added to `UNIFIED_SOURCES`. + +### 3. IDL + +- Duplicate the public API (including its docs) to [`xpcom/nsIGleanMetrics.idl`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/xpcom/nsIGleanMetrics.idl) with the name `nsIGleanX` (e.g. `nsIGleanCounter`). + - Inherit from `nsISupports`. + - The naming style for members here is `lowerCamelCase`. You'll need a [GUID](https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Generating_GUIDs) because this is XPCOM, but you'll only need the canonical form since we're only exposing to JS. + - The `testGetValue` method will return a `jsval`. + +### 4. JS Impl + +- Add an `nsIGleanX`-deriving, `XMetric`-owning type called `GleanX` (e.g. `GleanCounter`) in the same header and `.cpp` as `XMetric` in [`bindings/private/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/bindings/private/). + - Don't declare any methods beyond a ctor (takes a `uint32_t` metric id, init-constructs a `impl::XMetric` member) and dtor (`default`): the IDL will do the rest so long as you remember to add `NS_DECL_ISUPPORTS` and `NS_DECL_NSIGLEANX`. + - In the definition of `GleanX`, member identifiers are back to `CamelCase` and need macros like `NS_IMETHODIMP`. Delegate operations to the owned `XMetric`, returning `NS_OK` no matter what in non-test methods. + - Test-only methods can return `NS_ERROR` codes on failures, but mostly return `NS_OK` and use `undefined` in the `JS::MutableHandleValue` result to signal no value. + +### 6. Tests + +Two languages means two test suites. + +- Add a never-expiring test-only metric of your type to [`test_metrics.yaml`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/test_metrics.yaml). + - Feel free to be clever with the name, but be sure to make clear that it is test-only. +- **C++ Tests (GTest)** - Add a small test case to [`gtest/TestFog.cpp`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/gtest/TestFog.cpp). + - For more details, peruse the [testing docs](testing.md). +- **JS Tests (xpcshell)** - Add a small test case to [`xpcshell/test_Glean.js`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/xpcshell/test_Glean.js). + - For more details, peruse the [testing docs](testing.md). + +### 7. API Documentation + +TODO diff --git a/toolkit/components/glean/docs/preferences.md b/toolkit/components/glean/docs/preferences.md new file mode 100644 index 0000000000..1a0efffc10 --- /dev/null +++ b/toolkit/components/glean/docs/preferences.md @@ -0,0 +1,21 @@ +# Preferences + +## User Preferences + +`datareporting.healthreport.uploadEnabled` + +This determines whether the Glean SDK is enabled. +It can be controlled by users via `about:preferences#privacy`. +If this is set to false from true, we send a +["deletion-request" ping](https://mozilla.github.io/glean/book/user/pings/deletion_request.html) +and no data collections will be persisted or reported from that point. + +## Test-only Preferences + +`telemetry.fog.test.localhost_port` + +If set to a value `port` which is greater than 0, pings will be sent to +`http://localhost:port` instead of `https://incoming.telemetry.mozilla.org`. +If set to a value which is less than 0, FOG will not squelch all sent pings, +telling the Glean SDK that the ping was sent successfully. +Defaults to 0. diff --git a/toolkit/components/glean/docs/storage.md b/toolkit/components/glean/docs/storage.md new file mode 100644 index 0000000000..00464c6173 --- /dev/null +++ b/toolkit/components/glean/docs/storage.md @@ -0,0 +1,14 @@ +# Storage + +Both FOG and the Glean SDK require some storage in the +[Firefox Profile Directory](https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Multiple_profiles). + +## FOG + +At present FOG's storage is limited to its [preferences](preferences.md). + +## Glean SDK + +The Glean SDK stores things in the +[Glean Data Directory](https://mozilla.github.io/glean/book/dev/core/internal/directory-structure.html) +which can be found at `<profile_dir>/datareporting/glean`. diff --git a/toolkit/components/glean/docs/style_guide.md b/toolkit/components/glean/docs/style_guide.md new file mode 100644 index 0000000000..6f37f23a8a --- /dev/null +++ b/toolkit/components/glean/docs/style_guide.md @@ -0,0 +1,58 @@ +# FOG Documentation Style Guide + +FOG's Documentation is written in Markdown. +You can find its source at `toolkit/components/glean/docs`. + +## Line breaks + +We will use [semantic linefeeds]: +* Break anywhere before 80-100 characters +* Break after any punctuation when it makes sense +* Break before or after any markdown when it makes sense + +**Tip:** To keep lines narrow, use markdown's [reference link] +feature whenever it makes sense (or all the time. Up to you.). + +## Linking to other documentation + +Linking to other external documentation is [easy][reference link]. +Linking to other pieces of documentation in the source docs might not be. + +To link to another markdown page in FOG's documentation, you can use +```md +[link text](page_name.md) +``` + +Sphinx will automagically transform that to an +appropriately-base-url'd url with a `.html` suffix. + +Unfortunately, this doesn't work for linking to +`.rst` files like those in use in [Telemetry]'s documentation. +(Follow [bug 1621950] for updates). + +In those cases you have to link it like it's html. +For example, to link to [Telemetry] you can use either of +```md +[Telemetry](../telemetry) +[Telemetry](../telemetry/index.html) +``` + +Both will work. Both will generate warnings. +For example, the first form will generate this: +```console +None:any reference target not found: ../telemetry +``` +But it will still work because linking to a directory in html links to its +`index.html` (which is where `index.rst` ends up). + +We can suppress this by putting a fake anchor +(like `#https://`) on the end to fool Sphinx into not checking it. +But that seems like a hack more unseemly than the warnings, +so let's not. + + +[semantic linefeeds]: https://rhodesmill.org/brandon/2012/one-sentence-per-line/ +[reference link]: https://spec.commonmark.org/0.29/#reference-link +[Telemetry]: ../telemetry +[#firefox-source-docs:mozilla.org]: https://chat.mozilla.org/#/room/#firefox-source-docs:mozilla.org +[bug 1621950]: https://bugzilla.mozilla.org/show_bug.cgi?id=1621950 diff --git a/toolkit/components/glean/docs/testing.md b/toolkit/components/glean/docs/testing.md new file mode 100644 index 0000000000..2ef6ace0f6 --- /dev/null +++ b/toolkit/components/glean/docs/testing.md @@ -0,0 +1,117 @@ +# Testing + +Given the multiple API languages, processes, and dependencies, +testing FOG is a matter of choosing the right tool for the situation. + +## Logging + +An often-overlooked first line of testing is "what do the logs say?". +To turn on logging for FOG, use any of the following: +* Run Firefox with `RUST_LOG="fog_control,fog,glean_core"`. + * On some platforms this will use terminal colours to indicate log level. +* Run Firefox with `MOZ_LOG="timestamp,glean::*:5,fog::*:5,fog_control::*:5,glean_core::*:5"`. +* Set the following prefs: + * `logging.config.timestamp` to `true` + * `logging.fog_control::*` to `5` + * `logging.fog::*` to `5` + * `logging.glean_core::*` to `5` + +For more information on logging in Firefox Desktop, see the +[Gecko Logging docs](https://developer.mozilla.org/docs/Mozilla/Developer_guide/Gecko_Logging). + +## `about:glean` + +`about:glean` is a page in a running Firefox that allows you to +[debug the Glean SDK](https://mozilla.github.io/glean/book/user/debugging/index.html) +in Firefox Desktop. +It does this through the displayed user interface (just follow the instructions). + +## Rust + +Not all of our Rust code can be tested in a single fashion, unfortunately. + +### Using `rusttests` + +If the crate you're testing has no Gecko symbols you can write standard +[Rust tests](https://doc.rust-lang.org/book/ch11-01-writing-tests.html). + +This supports both unit tests +(inline in the file under test) and integration tests +(in the `tests/` folder in the crate root). +Metric type tests are currently written as unit tests inline in the file, +as they require access to the metric ID, which should only be exposed in tests. + +To run FOG's `rusttests` suite use `mach rusttests` + +If the crate uses only a few Gecko symbols, they may use the +`with_gecko` feature to conditionally use them. +This allows the crate to test its non-Gecko-adjacent code using Rust tests. +(You will need to cover the Gecko-adjacent code via another means.) + +### Using `gtest` + +Because Gecko symbols aren't built for the +`rusttests` build, +any test that is written for code that uses Gecko symbols should be written as a +[`gtest`](https://github.com/google/googletest) +in `toolkit/components/glean/gtest/`. +You can write the actual test code in Rust. +It needs to be accompanied by a C++ GTest that calls a C FFI-exported Rust function. +See [Testing & Debugging Rust Code](/testing-rust-code/) for more. +See [`toolkit/components/glean/gtest/TestFog.cpp`](https://searchfox.org/mozilla-central/source/toolkit/components/glean/gtest/TestFog.cpp) +and [`toolkit/components/glean/gtest/test.rs`](https://searchfox.org/mozilla-central/source/toolkit/components/glean/gtest/test.rs) +for an example. + +By necessity these can only be integration tests against the compiled crate. + +**Note:** When adding a new test file, don't forget to add it to +`toolkit/components/glean/gtest/moz.build` and use the +`FOG` prefix in your test names +(e.g. `TEST(FOG, YourTestName) { ... }`). + +To run FOG's Rust `gtest` suite use `mach gtest FOG.*` + +## Python + +The [Glean Parser](https://github.com/mozilla/glean_parser/) +has been augmented to generate FOG-specific APIs for Glean metrics. +This augmentation is tested by running: + +`mach test toolkit/components/glean/pytest` + +These tests require Python 3+. +If your default Python is Python 2, you may need to instead run: + +`python3 mach python-test toolkit/components/glean/pytest` + +## C++ + +To test the C++ parts of FOG's implementation +(like metric types) +you should use `gtest`. +FOG's `gtest` tests are in +[`gtest/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/gtest/). + +You can either add a test case to an existing file or add a new file. +If you add a new file, remember to add it to the +[`moz.build`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/gtest/moz.build)) +or the test runner won't be able to find it. + +All tests should start with `FOG` so that all tests are run with +`./mach gtest FOG*`. + +## JS + +To test the JS parts of FOG's implementation +(like metric types) +you should use `xpcshell`. +FOG's `xpcshell` tests are in +[`xpcshell/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/xpcshell). + +You can either add a test case to an existing file or add a new file. +If you add a new file, remember to add it to the +[`xpcshell.ini`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/xpcshell/xpcshell.ini) +or the test runner will not be able to find it. + +To run FOG's JS tests, run: +`./mach test toolkit/components/glean/xpcshell` diff --git a/toolkit/components/glean/docs/updating_parser.md b/toolkit/components/glean/docs/updating_parser.md new file mode 100644 index 0000000000..6603090279 --- /dev/null +++ b/toolkit/components/glean/docs/updating_parser.md @@ -0,0 +1,37 @@ +# Updating glean_parser + +Project FOG uses the `glean_parser` to generate code from metric definitions. +It depends on [glean-parser] from pypi.org + +[glean-parser]: https://pypi.org/project/glean-parser/ + +To update the in-tree glean-parser run + +``` +./mach vendor python glean_parser==M.m.p +``` + +where `M.m.p` is the version number, e.g. 1.28.0. + +```eval_rst +.. note:: + + **Important**: the glean_parser and all of its dependencies must support Python 3.5, as discussed here. + This is the minimum version supported by mach and installed on the CI images for running tests. + This is enforced by the version ranges declared in the Python installation manifest. +``` + +## Version mismatch of the Python dependencies + +The logic for handling version mismatches is very similar to the one for the Rust crates. +See [Updating the Glean SDK](updating_sdk.html) for details. +However, updating Python packages also requires to think about Python 3.5 (and Python 2, still) compatibility. + +## Keeping versions in sync + +The Glean SDK and `glean_parser` are currently released as separate projects. +However each Glean SDK release requires a specific `glean_parser` version. +When updating one or the other ensure versions stay compatible. +You can find the currently used `glean_parser` version in the Glean SDK source tree, e.g. in [sdk_generator.sh]. + +[sdk_generator.sh]: https://github.com/mozilla/glean/blob/main/glean-core/ios/sdk_generator.sh#L28 diff --git a/toolkit/components/glean/docs/updating_sdk.md b/toolkit/components/glean/docs/updating_sdk.md new file mode 100644 index 0000000000..70752c980b --- /dev/null +++ b/toolkit/components/glean/docs/updating_sdk.md @@ -0,0 +1,39 @@ +# Updating the Glean SDK + +Project FOG uses the published Glean SDK. +It currently depends on [glean-core] from crates.io. + +[glean-core]: https://crates.io/crates/glean-core + +To update the dependency: + +1. Bump the version of the `glean-core` crate in and `toolkit/components/glean/Cargo.toml` and `toolkit/components/glean/api/Cargo.toml`. +2. Run `mach vendor rust`. This fetches all dependencies and adds them to `mozilla-central/third_pary/rust`. + +## Version mismatches of Rust dependencies + +Other crates that are already vendored might require a different version of the same dependencies that the Glean SDK requires. +The general strategy for Rust dependencies is to keep one single version of the dependency in-tree +(see [comment #8 in bug 1591555](https://bugzilla.mozilla.org/show_bug.cgi?id=1591555#c8)). +This might be hard to do in reality since some dependencies might require tweaks in order to work. +The following strategy can be followed to decide on version mismatches: + +* If the versions only **differ by the patch version**, Cargo will keep the vendored version, + unless some other dependency pinned specific patch versions; + assuming it doesn’t break the Glean SDK; + if it does, follow the next steps; +* If the version of the **vendored dependency is newer** (greater major or minor version) than the version required by the Glean SDK, + [file a bug in the Glean SDK component][glean-bug] to get Glean to require the same version; +* If the version of the **vendored dependency is older** (lower major or minor version), consider updating the vendored version to the newer one; + seek review from the person who vendored that dependency in the first place; + if that is not possible or breaks mozilla-central build, then consider keeping both versions vendored in-tree; please note that this option will probably only be approved for small crates. + +## Keeping versions in sync + +The Glean SDK and `glean_parser` are currently released as separate projects. +However each Glean SDK release requires a specific `glean_parser` version. +When updating one or the other ensure versions stay compatible. +You can find the currently used `glean_parser` version in the Glean SDK source tree, e.g. in [sdk_generator.sh]. + +[sdk_generator.sh]: https://github.com/mozilla/glean/blob/main/glean-core/ios/sdk_generator.sh#L28 +[glean-bug]: https://bugzilla.mozilla.org/enter_bug.cgi?product=Data+Platform+and+Tools&component=Glean%3A+SDK&priority=P3&status_whiteboard=%5Btelemetry%3Aglean-rs%3Am%3F%5D |
