@adguard/agtree

 

Universal adblock filter list parser

Supported syntaxes:

AdGuard | uBlock Origin | AdBlock | Adblock Plus

Table of Contents:

• Introduction
• Usage
  • Parsing rules
  • Parsing filter lists
• Development & Contribution
• Ideas & Questions
• License
• References

Introduction

AGTree is a universal adblock filter list parser. It supports all syntaxes currently in use: AdGuard, uBlock Origin and AdBlock / Adblock Plus.

Usage

AGTree provides a powerful, error-tolerant parser for all kinds of ad blocking rules. It fully supports AdGuard, uBlock Origin and Adblock Plus syntaxes, and provides a high-detail AST for all rules.

Basically, the parser API has two main parts:

• Parser: parsing rules (string → AST)
• Generator: serialization of ASTs (AST → string)

Parsing rules

You can parse individual rules using the RuleParser class. It has a parse method that takes a rule string and returns an AST. For example, this code:

import { RuleParser } from "@adguard/agtree";

// RuleParser automatically determines the rule type
const ast = RuleParser.parse("/ads.js^$script");

will gives you this AST:

{
  "type": "NetworkRule",
  "category": "Network",
  "syntax": "Common",
  "exception": false,
  "pattern": {
    "type": "Value",
    "value": "/ads.js^"
  },
  "modifiers": {
    "type": "ModifierList",
    "children": [
      {
        "type": "Modifier",
        "modifier": {
          "type": "Value",
          "value": "script"
        },
        "exception": false
      }
    ]
  }
}

Show full AST (with locations)

{
  "type": "NetworkRule",
  "loc": {
    "start": {
      "offset": 0,
      "line": 1,
      "column": 1
    },
    "end": {
      "offset": 15,
      "line": 1,
      "column": 16
    }
  },
  "raws": {
    "text": "/ads.js^$script"
  },
  "category": "Network",
  "syntax": "Common",
  "exception": false,
  "pattern": {
    "type": "Value",
    "loc": {
      "start": {
        "offset": 0,
        "line": 1,
        "column": 1
      },
      "end": {
        "offset": 8,
        "line": 1,
        "column": 9
      }
    },
    "value": "/ads.js^"
  },
  "modifiers": {
    "type": "ModifierList",
    "loc": {
      "start": {
        "offset": 9,
        "line": 1,
        "column": 10
      },
      "end": {
        "offset": 15,
        "line": 1,
        "column": 16
      }
    },
    "children": [
      {
        "type": "Modifier",
        "loc": {
          "start": {
            "offset": 9,
            "line": 1,
            "column": 10
          },
          "end": {
            "offset": 15,
            "line": 1,
            "column": 16
          }
        },
        "modifier": {
          "type": "Value",
          "loc": {
            "start": {
              "offset": 9,
              "line": 1,
              "column": 10
            },
            "end": {
              "offset": 15,
              "line": 1,
              "column": 16
            }
          },
          "value": "script"
        },
        "exception": false
      }
    ]
  }
}

As you can see, this AST is very detailed and contains all the information about the rule: syntax, category, exception, modifiers, node locations, and so on. Locations are especially useful for linters, since they allow you to point to the exact place in the rule where the error occurred.

And this code:

RuleParser.generate(ast);

will generate the adblock rule from the AST (serialization):

/ads.js^$script

Please keep in mind that the parser omits unnecessary whitespaces, so the generated rule may not match with the original rule character by character. Only the formatting can change, the rule itself remains the same. You can pass any rule to the parser, it automatically determines the type and category of the rule. If the rule is syntactically incorrect, the parser will throw an error.

Parsing filter lists

You can also parse complete filter lists using the FilterListParser class. It works the same way as the RuleParser class. Here is an example of parsing EasyList and generating it back:

import { FilterListParser } from "@adguard/agtree";
import { writeFile } from "fs/promises";
// Requires installing "node-fetch" package
import fetch from "node-fetch";

// Download EasyList
const easyList = await (
  await fetch("https://easylist.to/easylist/easylist.txt")
).text();

// Or read it from file
// const easyList = await readFile('easylist.txt', 'utf-8');

// Parse EasyList to AST. By default, parser is very tolerant,
// if it can't parse some rules, it will just mark them as "raw".
// If you want to disable this behavior, you can pass the second
// argument as "false" to the "parse" method, like this:
// const ast = FilterListParser.parse(easyList, false);
const ast = FilterListParser.parse(easyList);

// Generate filter list from filter list AST
const easyListGenerated = FilterListParser.generate(ast);

// Write generated filter list to file
await writeFile("easylist-generated.txt", easyListGenerated);

Development & Contribution

Please read the CONTRIBUTING.md file for details on how to contribute to this project.

Ideas & Questions

If you have any questions or ideas for new features, please open an issue or a discussion. We will be happy to discuss it with you.

License

AGTree is licensed under the MIT License. See the LICENSE file for details.

References

Here are some useful links to help you write adblock rules. This list is not exhaustive, so if you know any other useful resources, please let us know.

• Syntax documentation:
  • AdGuard: How to create your own ad filters
  • uBlock Origin: Static filter syntax
  • Adblock Plus: How to write filters
• Extended CSS documentation:
  • MDN: CSS selectors
  • AdGuard: Extended CSS capabilities
  • uBlock Origin: Procedural cosmetic filters
  • Adblock Plus: Extended CSS selectors
• Scriptlets:
  • AdGuard scriptlets
  • uBlock Origin scriptlets
  • Adblock Plus snippets
• Third party libraries:
  • CSSTree docs
• AdGuard's compatibility table