You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While reading the paper [1] I saw that there's a potential for an experiment re-run currently not exactly supported by Excos.
Let's take 10% of users into an experiment
50/50 control and treatment
Run experiment
Now we want to use the same users, but re-randomize the variant assignment
Currently the first part would be achieved with a Feature config with two variants at [0;0.05) and [0.05;0.1). To change randomization we need to alter the salt, but that will re-randomize the entire population, choosing a different set of 10% users.
To address it we need a feature level population constraint.
I've already added variant-level salts to support GrowthBook, so they can be reused to execute re-randomization with the selected user group at a feature level.
The text was updated successfully, but these errors were encountered:
manio143
changed the title
Two-level hashing
Two-level randomization (1-select population, 2-split across variants)
Mar 12, 2024
While reading the paper [1] I saw that there's a potential for an experiment re-run currently not exactly supported by Excos.
Currently the first part would be achieved with a Feature config with two variants at
[0;0.05)
and[0.05;0.1)
. To change randomization we need to alter the salt, but that will re-randomize the entire population, choosing a different set of 10% users.To address it we need a feature level population constraint.
I've already added variant-level salts to support GrowthBook, so they can be reused to execute re-randomization with the selected user group at a feature level.
The text was updated successfully, but these errors were encountered: