@appannie/ab-testing
Advanced tools
The AB Testing library implements multi-variance testing with a deterministic algorithm. It doesn't require any complex backend or database. Supporting both Javascript and Python.
The AB Testing library segmentating users under different "cohorts", so you could use an if-else-like statement to implement different features base on the segmentation.
There're 2 ways of segmentation:
allocation_criteria. Take a look at this example: first 25% users, it will always result into the same group of users in experiment A, but results in another completely different group of users for experiment B.The segmentation decisions are made base on a configuration file. The file should be json formated and hosted centralized place. More details about the configuration file below.
The config file can be optionaly hashed so the content is not reversable. In this case you can host the file publicly, for example, on you CDN server, or in a publich S3 bucket.
Experiment: A subject that you want to do ab-testing with. You can have multiple experiments on going at the same time, the library can handle them independently.
Cohort: Inside an experiment, users are classified into groups. The groups are called Cohorts.
control: The name of the default cohort. Which is, the cohort returned when the user doesn't matche any rule on any cohorts, or the experiment just doesn't exists.
@appannie/react-ab-testing
React binding for the vanilla Javascript package, featuring similar functionality.
npm install @appannie/react-ab-testing
# or
yarn add @appannie/react-ab-testing
Wrap your app with the ABTestingController
import { ABTestingController } from '@appannie/react-ab-testing';
const MyApp = ({ user }) => {
const profile = {
persona: user.persona,
employee: user.isEmployee,
};
return (
<ABTestingController config={testConfig} userId={user.id} userProfile={profile}>
<App />
</ABTestingController>
);
};
The required props are:
config: the configuration object.userId: a unique identifier for your current user. This ID should be the same across visits to make sure your user always end up in the same cohorts. It can a string or a number.userProfile: a key/value map used to force include a user in given cohorts.Then within your app, check the cohort a user is assigned to using the useCohortOf hook.
import { useCohortOf } from '@appannie/react-ab-testing';
const Component = () => {
const cohort = useCohortOf('experiment-name');
switch (cohort) {
case 'blue':
return <BlueButton />;
case 'red':
return <RedButton />;
// 'control' is the default cohort. All experiments have a control cohort.
case 'control':
default:
return <Default />;
}
};
@appannie/ab-testing
Core Javascript package that implementing ab-testing SDK APIs.
npm install @appannie/ab-testing
# or
yarn add @appannie/ab-testing
import { Experiments } from '@appannie/ab-testing';
const profile = {
persona: user.persona,
employee: user.isEmployee,
};
const experiments = new Experiments(config, user.id, profile);
const cohort = experiments.getCohort('experiment-name');
switch (cohort) {
case 'blue':
console.log('user in the blue cohort');
break;
case 'red':
console.log('user in the red cohort');
break;
// 'control' is the default cohort. All experiments have a control cohort.
case 'control':
default:
console.log('user in the control (default) cohort');
break;
}
@appannie/ab-testing-hash-object
An helper library that implements hashing for the configurations (user profile and the configuration force_include section) in case you want to host the config file publicly but still keep the content private, or just don't want to leave PIIs been transmitted in plain text.
The hashing algorithm used is sha256, and we're using a universal implementation (create-hash) that'll work both in Node and the browser.
npm install @appannie/ab-testing-hash-object
# or
yarn add @appannie/ab-testing-hash-object
import { hashObject } from '@appannie/ab-testing-hash-object';
const profile = hashObject(
{
persona: user.persona,
employee: user.isEmployee,
},
salt
);
const forceIncludeCondition = hashObject(
{
persona: ['data analyst'],
employee: ['yes'],
},
salt
);
You'll want to make sure the salt is the same on your server and your client for the encoding result to be the same.
py-ab-testing
Python implementation for the same APIs in @appannie/ab-testing and @appannie/ab-testing-hash-object
pip install py-ab-testing
# or
pipenv install py-ab-testing
Note: The config variable holds an dict with configuration file format that documented here.
from ABTesting import ABTestingController
user_profile = {
'persona': user.persona,
'employee': user.isEmployee,
}
controller = ABTestingController(config, user.id, user_profile)
cohort = controller.get_cohort('experiment-name')
if cohort == 'blue':
do_something()
elif cohort == 'red':
do_something_else()
else:
do_default_behavior()
Similar to the Javascript SDKs, the package comes with an optional util for hashing private information with sha256.
from ABTesting.utils import hash_dict
config['salt'] = salt
for experiment in config['experiments']:
for cohort in experiment['cohorts']:
cohort['force_include'] = hash_dict(cohort['force_include'], salt)
from ABTesting.utils import hash_dict
hashed_user_profile = hash_dict(
{
'persona': user.persona,
'employee': user.isEmployee,
},
salt
)
# Make sure config is hashed with the same salt
controller = ABTestingController(config, user.id, hashed_user_profile)
{
// The version isn't used right now, it is used to allow potentially breaking configuration
// change in the future. Current version is "1.0"
version: "1.0",
experiments: [
{
// Unique name used to refer to an experiment; used within `useCohortOf`
name: 'experiment-name',
cohorts: [
{
// Name of a cohort. These are the values returned by `useCohortOf`
name: 'blue',
// The force_include section is used to force users into given cohorts if their
// userProfile key. In many keys are used, *any* match will force include the user in
// the cohort.
// The force_include rules are checked in the order of the cohorts in the array and
// the first match wins.
// This section is optional.
force_include: {
persona: [
'data analyst'
]
},
// The users are allocated to values in a range of 0 to 100. The allocation config
// represents the slice of users allocated to a cohort. You can have multiple
// allocation ranges.
// Every range needs to be unique and not overlap other ranges in any other cohort.
allocation: [
[0, 25]
],
// The allocation_criteria key allows us to further filter the set of users by enforcing
// a criteria that must be valid before the cohort is approved. In this case,
// the user must have an email domain of data.ai or appannie.com.
allocation_criteria: {
email_domain: ['appannie.com', 'data.ai']
}
},
{
// "control" is the default cohort. All experiments always have a control cohort.
// All users not allocated to other cohorts will be assigned to "control" by default.
name: 'control'
},
]
}
]
}
At Data AI, we're maintaining this configuration in yaml format to reduce the syntax noise. We have an automated CI task to encode/encrypt and push the final configuration to a public S3 bucket from where our SDK retrieve the configuration.
You shouldn't list user private information (PII) within the configuration file as these are most likely public.
If you need to segment users using emails or information you wish/need to keep private, we suggest you to securely encrypt them.
We're providing an extra module @appannie/ab-testing-hash-object aiming to simplify this flow. This module will encode the keys and the values of your user profile. You then need to call it with the same salt while preprocessing your configuration and once when defining the user profile:
import { hashObject } from '@appannie/ab-testing-hash-object';
const profile = hashObject(
{
persona: user.persona,
employee: user.isEmployee,
},
config.salt
);
You can do this by force including a user or a group of users in the control cohort.
experiments:
- name: my_experiment
cohorts:
- name: control
force_include:
# Here we'd prevent any google.com users from seeing the experiment
email_domain:
- google.com
- name: cohort_A
allocation:
- [0, 10]
Force include users with email_domain equal to appannie.com under the desired cohort.
experiments:
- name: my_experiment
cohorts:
- name: control
- name: cohort_A
force_include:
# Force all Data AI user under the given cohort
email_domain:
- appannie.com
Made with ❤️ by Zhang Tao and Simon Boudrias from the Data AI Beijing office.
Available for public use under the MIT license.
FAQs
<!-- TOC -->
The npm package @appannie/ab-testing receives a total of 66 weekly downloads. As such, @appannie/ab-testing popularity was classified as not popular.
We found that @appannie/ab-testing demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 4 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.