@naturalcycles/abba

ABBA

A tool for generating and persisting AB test assignments

Table of Contents

1. Concepts
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Segmentation

Concepts

• Experiment: An individual experiment that will test a hypothesis
• Segmentation: The target audience for the experiment
• Sampling: Restrictions on what proportion of the target audience will be involved in the experiment
• Bucket: An allocation that defines what variant of a particular experience the user will have

Built With

• @naturalcycles/db-lib

(back to top)

Getting Started

Prerequisites

• A running MYSQL instance

Installation

Below is an example of how you can instruct your audience on installing and setting up your app. This template doesn't rely on any external dependencies or services.

1. Install NPM packages
   

   yarn add @naturalcyles/abba
   
   or
   
   npm install @naturalcyles/abba
   

2. Install the schema into your MySQL db instance using the migration script found here.

(back to top)

Usage

Create an instance of Abba

(Currently supports MySQL, probably all DocumentDBs but not verified.)

type AbbaConfig = {
  db: CommonDB // from @naturalcycles/db-lib
}

const abba = new Abba(config: AbbaConfig)

Create a new experiment

Creates a new experiment

async createExperiment(
	input: ExperimentInput,
	buckets: BucketInput[]
): Promise<Saved<Experiment>>

Update an experiment

Updates an existing experiment.

async updateExperiment(
	id: number,
	input: ExperimentInput,
	buckets: BucketInput[]
): Promise<Saved<Experiment>>

Delete an experiment

Delete an experiment. Removes all users assignments and buckets

async deleteExperiment(
	id: number
): Promise<void>

Get all existing user assignments

Gets all existing user assignments

async getAllExistingUserAssignments(
	userId: string
): Promise<Saved<UserAssignment>[]>

Get a users assignment

Get an assignment for a given user. If existingOnly is false, it will attempt generate a new assignment. segmentationData becomse required when existingOnly is false

async getUserAssignment(
  experimentId: number,
  userId: string,
  existingOnly: boolean,
  segmentationData?: SegmentationData,
): Promise<GeneratedUserAssignment | null>

Generate user assignments

Generate user assignments for all active experiments. Will return any existing assignments and attempt to generate new assignments.

async generateUserAssignments(
  userId: string,
  segmentationData: SegmentationData,
): Promise<GeneratedUserAssignment[]>

Getting assignment statistics

Get assignment statistics for an experiment.

async getExperimentAssignmentStatistics(
  experimentId: number
): Promise<AssignmentStatistics>

(back to top)

Segmentation

Experiments can be configured to target specific audiences using segmentation rules. When generating assignments it is possible to test these rules using user segmentation data which is an object containing key/value pairs unique to each user. (Allowed value types: string, number, boolean). A segmentation rule consist of the following properties:

  key: string, // the key of the corresponding segmentationData property.
  operator: '==' | '!=' | 'semver' | 'regex' | 'boolean', // the operator that will be used to execute the rule
  value: string | number | boolean, // the value the operator will be executed against

Segmentation rule operators

Equals (==)

Rule:

  { key: 'country', operator: '==', value: 'SE }

Example segmentation data:

{
  country: 'SE', // valid
  country: 'NO' // not valid
}

Not equals (!=)

Rule:

  { key: 'country', operator: '!=', value: 'SE' }

Example segmentation data:

{
  country: 'NO', // valid
  country: 'SE' // not valid
}

Boolean (boolean)

Rule:

  { key: 'isEligible', operator: 'boolean', value: true }

Example segmentation data:

{
  isEligible: true, // valid
  isEligible: false // not valid
}

Semver (semver)

Rule:

  { key: 'appVersion', operator: 'semver', value: '>1.1.0' }

Example segmentation data:

{
  appVersion: '1.2.0', // valid
  appVersion: '1' // not valid
}

Regex (regex)

Rule:

  { key: 'country', operator: 'regex', value: 'SE|NO' }

Example segmentation data:

{
  country: 'SE', // valid
  country: 'NO', // valid
  country: 'GB' // not valid
}

(back to top)