websitecategorization

Website / Domain Categorization API for Node.js

Website / Domain Categorization API is a Node.js module that uses machine learning model to classify arbitrary blocks of input texts or URLs into content categories.

Content categories are based on two taxonomies:

• Ecommerce (21 Tier 1 categories, 192 Tier 2 categories and 1113 Tier 3 categories)
• IAB based (29 Tier 1 categories, 447 Tier 2 categories)

Installation

npm i @websitecategorization/websitecategorization

Usage example

var request = require('request');
var options = {
  'method': 'POST',
  'url': 'https://www.websitecategorizationapi.com/api/gpt/gpt_category1.php',
  'headers': {
    'Content-Type': 'application/x-www-form-urlencoded'
  },
  form: {
    'query': 'earphone buds'
  }
};
request(options, function (error, response) {
  if (error) throw new Error(error);
  console.log(response.body);
});

Use cases

Web Categorization API is used by a wide variety of companies for many different use cases.

It is suitable for Ad Exchanges, Demand Side Platforms (DSPs), Supply Side Platforms (SSPs) and Ad Networks. SSP (Supply Side Platform) companies can e.g. use it to identify the advertiser’s category to check its eligibility for real-time bidding.

Other use cases include Web Content Filtering where a company can employ it to filter out non-work related websites, such as social media networks, shopping platforms and similar.

Website / Domain categorization API is based on a machine learning model that has been extensively tested and used both in small and large scale classification projects, including on a project with 30+ million texts.

It is continuously developed and regularly updated (in terms of training data set) to reflect and include new verticals arising each year.

UI Dashboard

Website categorization service can also be used in form of dashboard UI, as seen here:

Form of json

Example output from IAB1 Website Categorization API on example domain:

{
  "classification": [
    {
      "category": "Style & Fashion",
      "value": 0.6335134346543948
    },
    {
      "category": "Religion & Spirituality",
      "value": 0.31965677636420087
    },
    {
      "category": "Events and Attractions",
      "value": 0.028203161466589827
    },
    {
      "category": "Pop Culture",
      "value": 0.008486557302356994
    },
    {
      "category": "Books and Literature",
      "value": 0.0028975322143729425
    },
    {
      "category": "Shopping",
      "value": 0.0014989265842864407
    },
    {
      "category": "Fine Art",
      "value": 0.0014698938766846063
    },
    {
      "category": "Family and Relationships",
      "value": 0.0008695569530150543
    },
    {
      "category": "Hobbies & Interests",
      "value": 0.0007021051093678122
    },
    {
      "category": "Travel",
      "value": 0.00045551400716377827
    },
    {
      "category": "Movies",
      "value": 0.0003105774008160576
    },
    {
      "category": "Television",
      "value": 0.0002812439624312471
    },
    {
      "category": "Healthy Living",
      "value": 0.00027001968240167887
    },
    {
      "category": "Careers",
      "value": 0.0002666186301324818
    },
    {
      "category": "Food & Drink",
      "value": 0.0002460227720972317
    },
    {
      "category": "Home & Garden",
      "value": 0.00021331353597162862
    },
    {
      "category": "Medical Health",
      "value": 0.00018344636503169902
    },
    {
      "category": "Music and Audio",
      "value": 0.00007348860474246987
    },
    {
      "category": "Video Gaming",
      "value": 0.00006822010822593386
    },
    {
      "category": "Real Estate",
      "value": 0.00006517844821148466
    },
    {
      "category": "Pets",
      "value": 0.00006069812911973799
    },
    {
      "category": "Education",
      "value": 0.00004860296854985923
    },
    {
      "category": "News and Politics",
      "value": 0.000035123587801619264
    },
    {
      "category": "Sports",
      "value": 0.00003402965849228489
    },
    {
      "category": "Science",
      "value": 0.000026461875107857055
    },
    {
      "category": "Automotive",
      "value": 0.000024825949895016523
    },
    {
      "category": "Personal Finance",
      "value": 0.00001581204114251354
    },
    {
      "category": "Technology & Computing",
      "value": 0.000015037047929356491
    },
    {
      "category": "Business and Finance",
      "value": 0.000007820699466562138
    }
  ],
  "language": "en"
}

Curl

Supported API calls (in curl) that can be adapted to javascript:

curl --location --request POST 'https://www.websitecategorizationapi.com/api/gpt/gpt_category1.php' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'query=polaroid land camera' \
--data-urlencode 'api_key=b4dcde2ce5fb2d0b887b5e'

curl --location --request POST 'https://www.websitecategorizationapi.com/api/gpt/gpt_category2.php' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'query=polaroid land camera' \
--data-urlencode 'api_key=b4dcde2ce5fb2d0b887b5e'

curl --location --request POST 'https://www.websitecategorizationapi.com/api/gpt/gpt_category3.php' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'query=polaroid land camera' \
--data-urlencode 'api_key=b4dcde2ce5fb2d0b887b5e'

curl --location --request POST 'https://www.websitecategorizationapi.com/api/iab/gpt_category1.php' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'query=credit card' \
--data-urlencode 'api_key=b4dcde2ce5fb2d0b887b5e'

curl --location --request POST 'https://www.websitecategorizationapi.com/api/iab/gpt_category2.php' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'query=credit card' \
--data-urlencode 'api_key=b4dcde2ce5fb2d0b887b5e'

Support for languages

Service supports website categorization of texts written in german, french, italian, spanish, portuguese and many other languages.

Useful resources

• IAB taxonomy: https://iabtechlab.com/press-releases/tech-lab-releases-content-taxonomy-3-0/

• Facebook Taxonomy: https://www.facebook.com/business/help/526764014610932?id=725943027795860

• Introduction to Website Categorization

• Survey of text classification models: https://github.com/kk7nc/Text_Classification

• Introduction to product classification machine learning models: