@instantdomain/bandit
Advanced tools
Comparing version 1.0.14 to 1.0.15
{ | ||
"name": "@instantdomain/bandit", | ||
"version": "1.0.14", | ||
"version": "1.0.15", | ||
"license": "MIT", | ||
@@ -5,0 +5,0 @@ "repository": { |
# Instant Bandit | ||
Instant Bandit is a small set of React components and server-side helpers for authoring and running multi-armed bandit (MAB) experiments in websites and apps. Using an epsilon-greedy algorithm, Instant Bandit automatically presents multiple variants that you define to subsets of your traffic. The conversion rate of each variant is continuously measured, and the most successful variant is presented to the majority of your traffic. | ||
Instant Bandit is a small set of React components and server-side helpers for declaratively authoring multivariate tests. | ||
Using this library, defining and deploying variants is easy and requires little modification to your existing website or apps. | ||
Using an epsilon-greedy [Multi-armed bandit algorithm](https://en.wikipedia.org/wiki/Multi-armed_bandit), Instant Bandit automatically presents variants of your app/website to new visitors, and binds that variant to them on repeat visits. | ||
The conversion rate of each variant is continually measured, and the most successful variant at any given moment is assigned the highest probability to be chosen for new visitors. | ||
Using this library, defining and deploying variants is easy and requires little modification to existing websites or apps. | ||
You can find more in-depth documentation [here](./pages/docs/index.md). | ||
## A Simple Example | ||
Here's an example of an existing page: | ||
```TSX | ||
```tsx | ||
<Page> | ||
@@ -21,3 +27,3 @@ <Header>Welcome!</Header> | ||
```TSX | ||
```tsx | ||
<Page> | ||
@@ -29,2 +35,3 @@ <Header>Welcome!</Header> | ||
<InstantBandit> | ||
<Default> | ||
@@ -45,2 +52,3 @@ <SignUpButton /> | ||
</Variant> | ||
</InstantBandit> | ||
@@ -54,6 +62,9 @@ | ||
Note the use of the `<Default />` component. If metadata about the experiment and variants can't be loaded, the default component - the original button - will be displayed. Specifying a default is a good practice. | ||
Note the use of the `<Default />` component. | ||
If the current experiment ends, new visitors will see the default, i.e. the original. | ||
If metadata about the experiment and variants can't be loaded for some reason, such as a network error, the default will be used as a fallback. | ||
## How it Works | ||
When the Instant Bandit mounts, it immediately looks for a block of configuration called a "site". A _Site_ is a block of JSON that defines the experiments and variants. It looks like this: | ||
When the Instant Bandit component mounts, it looks for a block of configuration called a _Site_. A site is a block of JSON that defines the experiments and variants. It looks like this: | ||
@@ -65,3 +76,3 @@ ```JSON | ||
{ | ||
"id": "default", | ||
"id": "home-text-length", | ||
"variants": [ | ||
@@ -86,8 +97,13 @@ { | ||
A Site configuration describes the variants to test, and the probability that a visitor should see each one. When a new visitor views your site for the first time, one of these variants is chosen at random using the specified probability. Here, there is an 80% chance that a new user will see variant B. | ||
Experiments in a site have variants, and continually balance the probability that a new visitor should see each one. | ||
The probabilities for each variant are updated on the fly by the server, based on conversion rate. This is the "multi-armed bandit" part. If variant A's conversion rate begins exceeding variant B's conversion rate, the probabilities will be automatically updated, and A will begin to receive the majority of traffic. | ||
The probabilities for each variant are updated on the fly by the server, based on conversion rate, which is defined by you. | ||
Thanks to this, we can be sure that the best variant is consistently shown the most frequently, while still giving other variants the chance to shine. Rather than waste a significant portion of traffic on variants that don't resonate with visitors, such as in traditional A/B testing, Instant Bandit allows you to optimize conversions without wasting large amounts of impressions on things that don't stick. | ||
This is the "multi-armed bandit" part. If variant A's conversion rate begins exceeding variant B's conversion rate, the probabilities will be automatically updated, and A will begin to receive the majority of traffic. | ||
Thanks to this, we can be sure that the best variant is consistently shown the most frequently, while still giving other variants the chance to shine. | ||
Rather than waste a significant portion of traffic on variants that don't resonate with visitors, such as in traditional A/B testing, Instant Bandit allows you to optimize conversions without using up impressions on things that don't perform well. | ||
# Tracking Conversions | ||
@@ -97,19 +113,21 @@ In order to measure conversions and other metrics, Instant Bandit offers a convenient React hook: `useInstantBandit`. In our example, the code for `SignUpButton` can be augmented like so: | ||
```TS | ||
const { metrics } = useInstantBandit(); | ||
const { incrementMetric } = useInstantBandit(); | ||
// inside of the click handler: | ||
metrics.sinkEvent(ctx, "conversions"); | ||
// call this when a user converts | ||
incrementMetric(DefaultMetrics.CONVERSIONS); | ||
``` | ||
That's it! The `useInstantBandit` hook knows which variant is being displayed, and the "conversions" metric is automatically updated for the correct variant when a user hits the SignUpButton presented to them. | ||
That's it! The `useInstantBandit` hook knows which variant is being displayed, and the `conversions` metric is automatically updated for the correct experiment and variant when a visitor hits the SignUpButton presented to them. | ||
# The Backend | ||
Instant Bandit requires a backend exposing two endpoints: One to serve site configurations with probabilities inlined, and another to ingest metrics. By default, these are `/api/sites/[site name]` and `/api/metrics`. | ||
## The Backend | ||
Instant Bandit requires a backend exposing two endpoints: One to serve site configurations and variant probabilities, and another to ingest anonymous metrics. By default, these are `/api/sites/[site name]` and `/api/metrics`. | ||
This package includes helper functions to implement those endpoints in any Node.js-based web application, as well as an example of each implemented as Next.js API routes. See `/api/sites/[siteName].ts` and `/api/metrics.ts` in this repository. | ||
An instance of Redis is used as the default backend store for metrics, and one is configured in this repository. Implementors can replace Redis with their data store of choice, e.g. Postgres, GraphQL, MongoDB, etc. | ||
An instance of Redis is used as the default backend store for metrics, and one is configured in this repository. Implementers can easily replace Redis with their data store of choice, e.g. Postgres, GraphQL, MongoDB, etc. | ||
# Performance and SEO | ||
Instant Bandit loads quickly and minimizes or completely eliminates flickering and cumulative layout shift (CLS), both of which hurt the end user's experience and also can impact SEO. | ||
Instant Bandit loads quickly and minimizes or completely eliminates flickering and _Cumulative Layout Shift_ (CLS), both of which degrade the end user's experience and can impact SEO. | ||
@@ -126,5 +144,6 @@ Instant Bandit supports server-side rendering (SSR). In order to use SSR, e.g. in Next.js, the site configuration is obtained server-side, and passed in via properties using `getServerSideProps`. See `index.tsx` in this repo for an example. | ||
# Installation & Requirements | ||
## Installation & Requirements | ||
Clone this repo and run `yarn` to install dependencies and then `yarn dev` to run the development environment. | ||
The development environment requires _Docker Compose_, and will run a Redis server locally. |
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Major refactor
Supply chain riskPackage has recently undergone a major refactor. It may be unstable or indicate significant internal changes. Use caution when updating to versions that include significant changes.
Found 1 instance in 1 package
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Network access
Supply chain riskThis module accesses the network.
Found 1 instance in 1 package
Environment variable access
Supply chain riskPackage accesses environment variables, which may be a sign of credential stuffing or data theft.
Found 3 instances in 1 package
Filesystem access
Supply chain riskAccesses the file system, and could potentially read sensitive data.
Found 1 instance in 1 package
Minified code
QualityThis package contains minified code. This may be harmless in some cases where minified code is included in packaged libraries, however packages on npm should not minify code.
Found 3 instances in 1 package
1
142
0
1
10025
5
11