Skip to content
This repository has been archived by the owner on Jan 29, 2025. It is now read-only.

EXP-1865 add isRollout boolean property to Experiment type #179

Merged
merged 1 commit into from
Jan 12, 2022

Conversation

lmorchard
Copy link

Because:

  • we want to implement support for an isRollout flag in experiments

This commit:

  • adds an isRollout boolean to the NimbusExperiment type

  • introduces example recipes for both mobile and desktop for testing

@lmorchard
Copy link
Author

Kind of winging it on the example files as my first try. Let's see what I missed!

@lmorchard lmorchard force-pushed the exp-1855-add-isrollout branch 2 times, most recently from 91a5233 to 24e7fe2 Compare November 10, 2021 00:49
@lmorchard lmorchard mentioned this pull request Dec 1, 2021
30 tasks
@lmorchard lmorchard force-pushed the exp-1855-add-isrollout branch from 24e7fe2 to 3e06e21 Compare December 1, 2021 23:09
@lmorchard
Copy link
Author

lmorchard commented Dec 6, 2021

Seeing all the comments on the example data in this PR, it seems maybe worth backing up and seeing if we have a better source for these?

Maybe @heres-maria has a better way to come up with this sample data? That is - come up what is actually useful for testing these things, put that in this PR.

My goal was just to set isRollout: true with the rest of the experiment being unremarkable. Most of the rest of the properties are copied over from other data files in this repo, but it sounds like those are problematic also?

@heres-maria-sv-inactive
Copy link

heres-maria-sv-inactive commented Dec 7, 2021

Seeing all the comments on the example data in this PR, it seems maybe worth backing up and seeing if we have a better source for these?

Maybe @heres-maria has a better way to come up with this sample data? That is - come up what is actually useful for testing these things, put that in this PR.

My goal was just to set isRollout: true with the rest of the experiment being unremarkable. Most of the rest of the properties are copied over from other data files in this repo, but it sounds like those are problematic also?

@lmorchard For the feature part at least, maybe these production experiments can be used as the starting point for the iOS and Fenix recipes? They are current "rollouts".
https://experimenter.services.mozilla.com/nimbus/all-users-homescreen-ios-production
https://experimenter.services.mozilla.com/nimbus/all-users-home-screen-android-production

@lmorchard lmorchard force-pushed the exp-1855-add-isrollout branch from d880b64 to 593afd3 Compare December 9, 2021 20:05
Copy link
Author

@lmorchard lmorchard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, another stab at maybe some useful test files? Not sure if these are it.

Main thing is that I copied the recipes for these experiments and tried hand-tweaking them to add rollout properties as noted in comments:

"appName": "firefox_desktop",
"application": "firefox-desktop",
"arguments": {},
"isRollout": true,
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this new property, which is the main event in this PR.

],
"bucketConfig": {
"count": 1700,
"namespace": "urlbar-21-rollout",
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added -rollout to the namespace, to simulate the nimbus model logic here

"proposedEnrollment": 7,
"referenceBranch": "control",
"schemaVersion": "1.7.0",
"slug": "desktop-rollout-example-firefox-suggest-history-rollout-3",
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tweaked the slug and IDs

"proposedDuration": 28,
"proposedEnrollment": 7,
"referenceBranch": "control",
"schemaVersion": "1.7.0",
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea what the eventual schemaVersion should be, here. It depends on what PR lands first, and we'd like to remove it anyway per issue #186 - but we need something here to pass CI in the meantime

Copy link
Contributor

@piatra piatra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Desktop recipe looks good. Thanks.

@lmorchard lmorchard force-pushed the exp-1855-add-isrollout branch from 593afd3 to 7d94813 Compare December 13, 2021 22:14
@lmorchard
Copy link
Author

Alright, maybe third try's the charm? Rebased off main, copied the multifeature example data files, changed some names and added "isRollout": true to them along with adjusting the bucket namespace. Not sure what else needs doing?

@lmorchard lmorchard force-pushed the exp-1855-add-isrollout branch from 7d94813 to ef7c128 Compare December 14, 2021 19:14
"appName": "firefox_ios",
"application": "org.mozilla.ios.Firefox",
"arguments": {},
"branches": [

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has multiple branches and affects multiple features. Should this be single branch and single feature?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would also like to test the interaction between rollouts and a multifeature experiment of this type, but that would have isRollout false and a different namespace.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@heres-maria Can you compose the test data that would actually work for testing? We could add it to the repo after testing. I've tried a few times now in this PR - none of those tries has passed muster. Would rather just record what was used to successfully test the feature than keep guessing at what might work

Because:

* we want to implement support for an isRollout flag in experiments

This commit:

* adds an isRollout boolean to the NimbusExperiment type

* adds a test for boolean property schema validation for experiments
@lmorchard lmorchard force-pushed the exp-1855-add-isrollout branch from ef7c128 to 6f5d276 Compare January 4, 2022 20:40
@lmorchard
Copy link
Author

Revised this PR to drop the sample data. The schema change here is very small, but the testing data blocking merge of the PR seems to have been a moving target that I don't have the context to assemble properly.

Maybe we should just add the sample data after successful testing? That way we record what was used rather than trying to guess what might work?

@lmorchard
Copy link
Author

Missed the thumbs up earlier, just going to merge this and we can move on with sample data when we have it

@heres-maria-sv-inactive

Possible testing data:
Desktop:

Desktop Rollout with pocketNewtab enabled
{
  "slug": "firefox-desktop-rollout-test-final",
  "appId": "firefox-desktop",
  "appName": "firefox_desktop",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "feature": {
        "value": {},
        "enabled": true,
        "featureId": "unused-feature-id-for-legacy-support"
      },
      "features": [
        {
          "value": {
            "loadMore": true,
            "compactLayout": true,
            "newFooterSection": true,
            "lastCardMessageEnabled": true
          },
          "enabled": true,
          "featureId": "pocketNewtab"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": true,
  "probeSets": [],
  "startDate": "2021-10-26",
  "targeting": "true",
  "featureIds": [
    "pocketNewtab"
  ],
  "application": "firefox-desktop",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "firefox-desktop-pocketNewtab-release-rollout",
    "randomizationUnit": "normandy_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Firefox Desktop Rollout Test Pocket",
  "referenceBranch": "control",
  "proposedDuration": 63,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Rollout for pocketNewTab",
  "id": "firefox-desktop-rollout-test-final",
  "last_modified": 1639560912253
}
Desktop Rollout with privatebrowsing disabled
{
  "slug": "firefox-desktop-rollout-test-final-2",
  "appId": "firefox-desktop",
  "appName": "firefox_desktop",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "feature": {
        "value": {},
        "enabled": false,
        "featureId": "unused-feature-id-for-legacy-support"
      },
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "privatebrowsing"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": true,
  "probeSets": [],
  "startDate": "2021-10-26",
  "targeting": "true",
  "featureIds": [
    "privatebrowsing"
  ],
  "application": "firefox-desktop",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "firefox-desktop-privatebrowsing-release-rollout",
    "randomizationUnit": "normandy_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Firefox Desktop Rollout Test P",
  "referenceBranch": "control",
  "proposedDuration": 63,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Rollout for privatebrowsing",
  "id": "firefox-desktop-rollout-test-final-2",
  "last_modified": 1639560912253
}
Desktop Experiment with isRollout false pocketNewtab enabled
{
  "slug": "firefox-desktop-rollout-experiment1",
  "appId": "firefox-desktop",
  "appName": "firefox_desktop",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "feature": {
        "value": {},
        "enabled": true,
        "featureId": "unused-feature-id-for-legacy-support"
      },
      "features": [
        {
          "value": {
            "loadMore": true,
            "compactLayout": true,
            "newFooterSection": true,
            "lastCardMessageEnabled": true
          },
          "enabled": true,
          "featureId": "pocketNewtab"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": "2021-10-26",
  "targeting": "true",
  "featureIds": [
    "pocketNewtab"
  ],
  "application": "firefox-desktop",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "firefox-desktop-pocketNewtab-release-1",
    "randomizationUnit": "normandy_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Firefox Desktop Experiment 1 Pocket enabled",
  "referenceBranch": "control",
  "proposedDuration": 63,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment 1 enabled for pocketNewTab",
  "id": "firefox-desktop-rollout-experiment1",
  "last_modified": 1639560912253
}
Desktop Experiment with isRollout false pocketNewtab disabled
{
  "slug": "firefox-desktop-rollout-experiment2",
  "appId": "firefox-desktop",
  "appName": "firefox_desktop",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "feature": {
        "value": {},
        "enabled": false,
        "featureId": "unused-feature-id-for-legacy-support"
      },
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "pocketNewtab"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": "2021-10-26",
  "targeting": "true",
  "featureIds": [
    "pocketNewtab"
  ],
  "application": "firefox-desktop",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "firefox-desktop-pocketNewtab-release-2",
    "randomizationUnit": "normandy_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Firefox Desktop Experiment 2 Pocket disabled",
  "referenceBranch": "control",
  "proposedDuration": 63,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment 2 disabled for pocketNewTab",
  "id": "firefox-desktop-rollout-experiment2",
  "last_modified": 1639560912253
}
Desktop Multifeature Experiment with isRollout false pocketNewtab and privatebrowsing enabled
{
  "slug": "firefox-desktop-rollout-experimentm1",
  "appId": "firefox-desktop",
  "appName": "firefox_desktop",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "feature": {
        "value": {},
        "enabled": false,
        "featureId": "unused-feature-id-for-legacy-support"
      },
      "features": [
        {
          "value": {
            "loadMore": true,
            "compactLayout": true,
            "newFooterSection": true,
            "lastCardMessageEnabled": true
          },
          "enabled": true,
          "featureId": "pocketNewtab"
        },
        {
          "value": {
            "infoEnabled": true,
            "infoTitle": "Hello world",
            "infoBody": "This is some text",
            "infoLinkText": "This is a link",
            "infoLinkUrl": "http://foo.mozilla.com/%LOCALE%",
            "infoIcon": "chrome://branding/content/about-logo.png",
            "promoEnabled": true,
            "promoTitle": "Promo title",
            "promLinkText": "Promo link",
            "promoLinkUrl": "https://vpn.mozilla.org/?utm_source=firefox-browser&utm_medium=firefox-%CHANNEL%-browser&utm_campaign=private-browsing-vpn-link&utm_experiment=private-browsing-test&utm_branch=treatment"
          },
          "enabled": true,
          "featureId": "privatebrowsing"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": "2021-10-26",
  "targeting": "true",
  "featureIds": [
    "pocketNewtab",
    "privatebrowsing"
  ],
  "application": "firefox-desktop",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "firefox-desktop-pocketNewtab-privatebrowsing-release-1",
    "randomizationUnit": "normandy_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Experiment 1 multifeature enabled",
  "referenceBranch": "control",
  "proposedDuration": 63,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment 1 multifeature enabled",
  "id": "firefox-desktop-rollout-experimentm1",
  "last_modified": 1639560912253
}
Desktop Multifeature Experiment with isRollout false pocketNewtab and privatebrowsing disabled
{
  "slug": "firefox-desktop-rollout-experimentm2",
  "appId": "firefox-desktop",
  "appName": "firefox_desktop",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "feature": {
        "value": {},
        "enabled": false,
        "featureId": "unused-feature-id-for-legacy-support"
      },
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "pocketNewtab"
        },
        {
          "value": {},
          "enabled": false,
          "featureId": "privatebrowsing"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": "2021-10-26",
  "targeting": "true",
  "featureIds": [
    "pocketNewtab",
    "privatebrowsing"
  ],
  "application": "firefox-desktop",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "firefox-desktop-pocketNewtab-privatebrowsing-release-2",
    "randomizationUnit": "normandy_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Experiment 2 multifeature disabled",
  "referenceBranch": "control",
  "proposedDuration": 63,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment 2 multifeature disabled",
  "id": "firefox-desktop-rollout-experimentm2",
  "last_modified": 1639560912253
}

Firefox Android:

Android Rollout with homescreen enabled
{
  "slug": "firefox-android-rollout-test-final",
  "appId": "org.mozilla.firefox",
  "appName": "fenix",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {
            "sections-enabled": {
              "pocket": false,
              "topSites": false,
              "jumpBackIn": true,
              "recentlySaved": false,
              "recentExplorations": false
            }
          },
          "enabled": true,
          "featureId": "homescreen"
        }
      ]
    }
  ],
  "outcomes": [
    {
      "slug": "default-browser",
      "priority": "primary"
    }
  ],
  "arguments": {},
  "isRollout": true,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen"
  ],
  "application": "org.mozilla.firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "fenix-homescreen-release-rollout",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Homepage Android Rollout Test 1",
  "referenceBranch": "control",
  "proposedDuration": 45,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 28,
  "userFacingDescription": "Android ROllout Homescreen",
  "id": "firefox-android-rollout-test-final",
  "last_modified": 1639991916998
}
Android Rollout with homescreen disabled
{
  "slug": "firefox-android-rollout-test-final-2",
  "appId": "org.mozilla.firefox",
  "appName": "fenix",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "homescreen"
        }
      ]
    }
  ],
  "outcomes": [
    {
      "slug": "default-browser",
      "priority": "primary"
    }
  ],
  "arguments": {},
  "isRollout": true,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen"
  ],
  "application": "org.mozilla.firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "fenix-homescreen-release-rollout",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Homepage Android Rollout Test 2",
  "referenceBranch": "control",
  "proposedDuration": 45,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 28,
  "userFacingDescription": "Android Rollout Homescreen disabled",
  "id": "firefox-android-rollout-test-final-2",
  "last_modified": 1639991916998
}
Android Experiment with isRollout false homescreen enabled
{
  "slug": "firefox-android-homepage-experiment-1",
  "appId": "org.mozilla.firefox",
  "appName": "fenix",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {
            "sections-enabled": {
              "pocket": true,
              "topSites": false,
              "jumpBackIn": true,
              "recentlySaved": false,
              "recentExplorations": false
            }
          },
          "enabled": true,
          "featureId": "homescreen"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen"
  ],
  "application": "org.mozilla.firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "fenix-homescreen-release-3",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Homepage Android Experiment 1",
  "referenceBranch": "control",
  "proposedDuration": 45,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 28,
  "userFacingDescription": "Android Homepage on with a single difference",
  "id": "firefox-android-homepage-experiment-1",
  "last_modified": 1639991916998
}
Android Experiment with isRollout false homescreen disabled
{
  "slug": "firefox-android-homepage-experiment-2",
  "appId": "org.mozilla.firefox",
  "appName": "fenix",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "homescreen"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen"
  ],
  "application": "org.mozilla.firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "fenix-homescreen-release-4",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Homepage Android Experiment 2",
  "referenceBranch": "control",
  "proposedDuration": 45,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 28,
  "userFacingDescription": "Android Homepage off",
  "id": "firefox-android-homepage-experiment-2",
  "last_modified": 1639991916998
}
Android Multifeature Experiment with isRollout false homescreen and fenix-default-browser enabled
{
  "slug": "firefox-android-homepage-experiment-m1",
  "appId": "org.mozilla.firefox",
  "appName": "fenix",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "default_browser_settings_menu",
      "ratio": 1,
      "features": [
        {
          "value": {
            "sections-enabled": {
              "pocket": true,
              "topSites": false,
              "jumpBackIn": true,
              "recentlySaved": false,
              "recentExplorations": false
            }
          },
          "enabled": true,
          "featureId": "homescreen"
        },
        {
          "value": {},
          "enabled": true,
          "featureId": "fenix-default-browser"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "fenix-default-browser",
    "homescreen"
  ],
  "application": "org.mozilla.firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "fenix-fenix-default-browser-homescreen-release-3",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Homepage Android Experiment Multifeature 1",
  "referenceBranch": "default_browser_settings_menu",
  "proposedDuration": 45,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 28,
  "userFacingDescription": "Android Homepage on with a single difference multifeature",
  "id": "firefox-android-homepage-experiment-m1",
  "last_modified": 1639991916998
}
Android Multifeature Experiment with isRollout false homescreen and fenix-default-browser disabled
{
  "slug": "firefox-android-homepage-experiment-m2",
  "appId": "org.mozilla.firefox",
  "appName": "fenix",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "default_browser_settings_menu",
      "ratio": 1,
      "features": [
        {
          "value": {
            "sections-enabled": {
              "pocket": true,
              "topSites": false,
              "jumpBackIn": true,
              "recentlySaved": false,
              "recentExplorations": false
            }
          },
          "enabled": true,
          "featureId": "homescreen"
        },
        {
          "value": {},
          "enabled": true,
          "featureId": "fenix-default-browser"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "fenix-default-browser",
    "homescreen"
  ],
  "application": "org.mozilla.firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "fenix-fenix-default-browser-homescreen-release-3",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "Homepage Android Experiment Multifeature 2",
  "referenceBranch": "default_browser_settings_menu",
  "proposedDuration": 45,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 28,
  "userFacingDescription": "Android Homepage on with a single difference multifeature 2",
  "id": "firefox-android-homepage-experiment-m2",
  "last_modified": 1639991916998
}

Firefox iOS:

iOS Rollout with homescreen enabled
{
  "slug": "firefox-ios-rollout-test-final",
  "appId": "org.mozilla.ios.Firefox",
  "appName": "firefox_ios",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {
            "jumpBackIn": false,
            "libraryShortcuts": true,
            "recentlySaved": true
          },
          "enabled": true,
          "featureId": "homescreen"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": true,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen"
  ],
  "application": "org.mozilla.ios.Firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "ios-homescreen-release-rollout",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "iOS Rollout 1",
  "referenceBranch": "control",
  "proposedDuration": 28,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Rollout for homescreen enabled feature",
  "id": "firefox-ios-rollout-test-final",
  "last_modified": 1639561341272
}
iOS Rollout with homescreen disabled
{
  "slug": "firefox-ios-rollout-test-final-2",
  "appId": "org.mozilla.ios.Firefox",
  "appName": "firefox_ios",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "onboarding-default-browser"
        }
      ]
    }
  ],
  "outcomes": [
    {
      "slug": "default_browser",
      "priority": "primary"
    }
  ],
  "arguments": {},
  "isRollout": true,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "onboarding-default-browser"
  ],
  "application": "org.mozilla.ios.Firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "ios-onboarding-default-browser-release-rollout",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "iOS Rollout 2",
  "referenceBranch": "control",
  "proposedDuration": 28,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Rollout for onboarding-default-browser disabled feature",
  "id": "firefox-ios-rollout-test-final-2",
  "last_modified": 1639561341272
}
iOS Experiment with isRollout false homescreen enabled
{
  "slug": "firefox-ios-homepage-experiment-1",
  "appId": "org.mozilla.ios.Firefox",
  "appName": "firefox_ios",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {
            "jumpBackIn": false,
            "libraryShortcuts": true,
            "recentlySaved": true
          },
          "enabled": true,
          "featureId": "homescreen"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen"
  ],
  "application": "org.mozilla.ios.Firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "ios-homescreen-release-3",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "iOS Single Feature Experiment 1,
  "referenceBranch": "control",
  "proposedDuration": 28,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment for homescreen enabled feature",
  "id": "firefox-ios-homepage-experiment-1",
  "last_modified": 1639561341272
}
iOS Experiment with isRollout false homescreen disabled
  "slug": "firefox-ios-homepage-experiment-2",
  "appId": "org.mozilla.ios.Firefox",
  "appName": "firefox_ios",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "homescreen"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen"
  ],
  "application": "org.mozilla.ios.Firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "ios-homescreen-release-4",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "iOS Single Feature Experiment 2,
  "referenceBranch": "control",
  "proposedDuration": 28,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment for homescreen disabled feature",
  "id": "firefox-ios-homepage-experiment-2",
  "last_modified": 1639561341272
}
iOS Multifeature Experiment with isRollout false homescreen and fenix-default-browser enabled
{
  "slug": "firefox-ios-homepage-experiment-m1",
  "appId": "org.mozilla.ios.Firefox",
  "appName": "firefox_ios",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "default_browser_newtab_banner",
      "ratio": 1,
      "features": [
        {
          "value": {
            "jumpBackIn": false,
            "libraryShortcuts": true,
            "recentlySaved": true
          },
          "enabled": true,
          "featureId": "homescreen"
        },
        {
          "value": {},
          "enabled": true,
          "featureId": "onboarding-default-browser"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen",
    "onboarding-default-browser"
  ],
  "application": "org.mozilla.ios.Firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "ios-homescreen-onboarding-default-browser-release-3",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "iOS Single Feature Experiment Multifeature 1",
  "referenceBranch": "control",
  "proposedDuration": 28,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment multifeature for homescreen + onboarding-default-browser enabled features",
  "id": "firefox-ios-homepage-experiment-m1",
  "last_modified": 1639561341272
}
iOS Multifeature Experiment with isRollout false homescreen and fenix-default-browser disabled
{
  "slug": "firefox-ios-homepage-experiment-m2",
  "appId": "org.mozilla.ios.Firefox",
  "appName": "firefox_ios",
  "channel": "release",
  "endDate": null,
  "branches": [
    {
      "slug": "control",
      "ratio": 1,
      "features": [
        {
          "value": {},
          "enabled": false,
          "featureId": "homescreen"
        },
        {
          "value": {},
          "enabled": false,
          "featureId": "onboarding-default-browser"
        }
      ]
    }
  ],
  "outcomes": [],
  "arguments": {},
  "isRollout": false,
  "probeSets": [],
  "startDate": null,
  "targeting": "true",
  "featureIds": [
    "homescreen",
    "onboarding-default-browser"
  ],
  "application": "org.mozilla.ios.Firefox",
  "bucketConfig": {
    "count": 10000,
    "start": 0,
    "total": 10000,
    "namespace": "ios-homescreen-onboarding-default-browser-release-4",
    "randomizationUnit": "nimbus_id"
  },
  "schemaVersion": "1.6.2",
  "userFacingName": "iOS Single Feature Experiment Multifeature 2",
  "referenceBranch": "control",
  "proposedDuration": 28,
  "isEnrollmentPaused": false,
  "proposedEnrollment": 7,
  "userFacingDescription": "Experiment multifeature for homescreen + onboarding-default-browser disabled features",
  "id": "firefox-ios-homepage-experiment-m2",
  "last_modified": 1639561341272
}

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants