deprecation-crawler

deprecation-crawler

Preconditions

• Setup the target project.

Do this by checking out the target project e.g. RxJS

git clone [e-mail]:ReactiveX/rxjs.git rxjs`

• Navigate to the root of the repository.

cd rxjs

Usage

• In the root of the repository run the following command.

npx deprecation-crawler

Possible CLI params are:

• path (--path , -p ) Path to the deprecation-crawler.config.json. Pro Tip: use ./GIT_IGNORED_FOLDER/deprecation-crawler.config.json for a 'dry-run'.
• tag (--tag , -t ) Git tag to crawl. Pro Tip: use the current branch name for a 'dry-run' (git branch --show-current).

CLI Questions
The command will ask you a couple of questions regarding the included file locations and defaults to run the crawling and grouping process. It also includes helpful information.

Setup

Setup tsconfig (one-time)

• √ What's the location of the ts config file? By default ./tsconfig.json is set as tscinfig to extend from. These locations should represent the projects ts config settings used to determine the folders to crawl. Pro Tip: The result is stored in deprecation-crawler.tsconfig.json go there to edit included files.

Configuring the crawling process (one-time)

• √ What's the output directory? By default ./depercations is set. This will put the resulting data into the named folder. Pro Tip: Move it in a gitignored folder for a 'dry-run'.

• √ What's the deprecation keyword to look for? By default @deprecated is set. Looks for comments with this keyword to add to the deprecations list.

• √ What's the deprecation link to the docs (the deprecation ruid will be appended to this)? By default https://rxjs.dev/deprecations is set. Used to add a link in the sourcecode to this location. If a deprecation comment already has this link, it will be skipped.

Crawl

After that, the process should start crawling and every crawled file should get logged to the console. Looking for deprecations in path/to/file

• √ What git tag do you want to crawl? By default main is set. Pro Tip: Pass the tag name as cli param to save repetitive questions (npx deprecation-crawler --tag main)

For all crawled deprecations ruis get generated.

Sync Repository

• √ Do you want to sync the repo? This commits the changes to the version control Pro Tip: Edit the commit message under config#defaultCommitMessage

Group

After the crawling is done, a new message should show up in the console saying Adding grouping to deprecations...

You will get asked 2 questions for every crawled deprecation:

• √ Add group to deprecation path/to/file#LINE_NUMBER The text of the deprecation message gets listed and you have to enter a string for the group name. By default ungrouped is suggested. These strings serve as a reverence to the group.

• √ Add regexp to group This question asks for a regular expression used to check every new deprecation against it is to see if it matches the group's conditions/regexes. Every group can have multiple regular expressions to test a deprecation for.

The deprecation message as well as the passed reges string will get normalized

• trim white spaces
• transform multiple white space to one
• all lowercase

Examples for message The full deprecation message for {@link test} thingy!:

• Full message The full deprecation message for {@link test} thingy!
• Partial message/Includes full deprecation

Pro Tip: Use groups to:

• Address health checks e.g. deprecations without a message
• Maintain custom content for a group
• Validate the messages against a set of patterns

If you just hit enter no regular expression gets saved.

This means that every deprecation gets its own ruid generated by created from its function signature. Doing this enables us to detect already crawled deprecations, malicious deprecations as well as a ruid across machines and codebases/repositories.

After the deprecations have been processed, the source code of the repository will be updated. A link to the deprecation info will be added at the end of the deprecation message.

Format

There are currently 2 formatters built in:

• tag-based markdown
• group-based markdown

Both are set as default in the setup step.

• √ Update Formats? Hit enter to generate the registered formats.

Check if the generated configs ended up in ./[config#outputPath]/raw-deprecations.json. And the crawled deprecations are present in the configured folder (by default in the ./deprecations folder).

Pro Tip: edit the formatters under [config#formatters]

Usage during development

Run deprecation-crawler against the sandbox project

Run the command:

For npm:

npm run test

For yarn:

yarn test

This will:

• build the project
• setup the sandbox (copy input files as to-be crawled files)
• run the crawler against these files
• verify that: cli output is correct, deprecation notes are added to the repository, the output JSON file is correctly generated

Run deprecation-crawler against a real repository

• Build the deprecation finder:

For npm:

npm run build

For yarn:

yarn build

• Copy path to the packed file:

  Open the dist/packages/deprecation-crawler folder and search for the index.js file. copy the absolute path to index.js.

• Navigate to the root of your target project

cd path/to/the/root/folder

• Execute the packed file by using the absolute path copied in step 2.

npx path/to/deprecation-crawler/dist/index.js

• go on with the step 2 form section Usage