bulk-decaffeinate

bulk-decaffeinate

A tool, backed by decaffeinate, to help you convert some or all of a CoffeeScript codebase to JavaScript.

The tool can check a codebase for decaffeinate-readiness, and once the code (or a part of it) is ready, bulk-decaffeinate can actually run the conversion and some follow-up cleanups. Here's an example of checking the Hubot repo:

> npm install -g bulk-decaffeinate decaffeinate eslint
...
> git clone https://github.com/philc/vimium.git
...
> cd vimium
> bulk-decaffeinate check
Doing a dry run of decaffeinate on 50 files...
50/50
All checks succeeded! decaffeinate can convert all 50 files.
Run "bulk-decaffeinate convert" to convert the files to JavaScript.
> bulk-decaffeinate convert
Verifying that decaffeinate can successfully convert these files...
50/50
Backing up files to .original.coffee...
50/50
Renaming files from .coffee to .js...
50/50
Generating the first commit: "decaffeinate: Rename bg_utils.coffee and 49 other files from .coffee to .js"...
Moving files back...
50/50
Running decaffeinate on all files...
50/50
Deleting old files...
50/50
Setting proper extension for all files...
50/50
Generating the second commit: decaffeinate: Convert bg_utils.coffee and 49 other files to JS...
Running eslint --fix on all files...
50/50
[Skips eslint for all files because there is no config.]
Generating the third commit: decaffeinate: Run post-processing cleanups on bg_utils.coffee and 49 other files...
Successfully ran decaffeinate on 50 files.
You should now fix lint issues in any affected files.
All CoffeeScript files were backed up as .original.coffee files that you can use for comparison.
You can run "bulk-decaffeinate clean" to remove those files.
To allow git to properly track file history, you should NOT squash the generated commits together.

Assumptions

While the underlying decaffeinate tool tries to be general-purpose, bulk-decaffeinate intentionally makes some assumptions about your use case:

• Your build tooling can already handle JavaScript. Replacing a .coffee file with a .js file will "just work" as long as the files are equivalent.
• Adding some extra .original.coffee files as temporary backups won't cause trouble.
• You are using git for source control and all .coffee files being converted are already tracked in the git repo.
• You are using eslint for JS linting and you already have a .eslintrc file specifying your preferred styles.

Feel free to file an issue or submit a PR if these assumptions don't match your current project. Most steps shouldn't be hard to disable using a config setting.

What it does

bulk-decaffeinate supports a number of commands:

• check does a dry run of decaffeinate on the specified files and reports how decaffeinate-ready the set of files is.

• view-errors should be run after check reports failures. It opens the failed files in the online decaffeinate repl, with one browser tab per failed file. Each browser tab loads the online repl page with your source code encoded in the hash fragment of the URL. Because it is in the hash fragment and not a regular query param, your code is never sent to the server.

• convert actually converts the files from CofeeScript to JavaScript, generating a commit for each intermediate step.

• modernize-js runs only the JS-to-JS transformations on the specified JavaScript files. Unlike convert, this command does not create a git commit.

• clean deletes all files with ".original" in the name in the current directory or any of its subdirectories.

• land packages multiple commits into a merge commit based on an remote branch (origin/master by default). Splitting the decaffeinate work into separate commits allows git to properly track file history, but it can create added difficulty after code review is finished, and land helps with that. The land command does not actually push any commits; it just creates a merge commit that is ready to push after a sanity check.

  If the phabricatorAware option is set, the land command does extra work to make sure that every commit has a "Differential Revision" line and that the final merge commit has the commit description.

Here's what convert does in more detail:

• It does a dry run of decaffeinate on all files to make sure there won't be any failures.
• It backs up all .coffee files to .original.coffee files, which makes it easily to manually do a before-and-after comparison later.
• It generates a commit renaming the files from .coffee to .js (but not changing the contents). Putting this step in its own commit allows git to track the file history across renames (so, if possible, you should land the changes as a merge commit rather than squashing the commits together).
• It runs decaffeinate on all files and gets rid of the .coffee files, then generates a commit.
• If the jscodeshiftScripts config value is specified, it runs jscodeshift with those scripts in the order specified.
• If the mochaEnvFilePattern config value is specified, it prepends /* eslint-env mocha */ to the top of every test file.
• If the fixImportsConfig config value is specified, it runs a transform that does whole-codebase analysis to fix any import problems that might have been introduced by decaffeinate.
• It runs eslint --fix on all files, which applies some style fixes according to your lint rules. For any remaining lint failures, it puts a comment at the top of the file disabling those specific lint rules and leaves a TODO comment to fix any remaining style issues.
• If the codePrefix config value is specified, it prepends that string to every affected file.
• All post-decaffeinate changes are committed as a third commit.

In all generated commits, "decaffeinate" is used as the author name (but not the email address). This makes it clear to people using git blame that the file was generated using decaffeinate, and not necessarily authored by the person who happened to run the decaffeinate script.

If you want to see the full details, the source code should hopefully be fairly readable.

Configuration

You can specify custom configuration in a config file, usually called bulk-decaffeinate.config.js, in the current working directory. It should export a JS object with your config. Any file starting with bulk-decaffeinate and ending with .config.js will be counted, and multiple config files may exist at once. If there are multiple config files, they are merged, with alphabetically-later config file names taking precedence over alphabetically-earlier files.

Alternatively, you may specify the config file location using the --config option, e.g. bulk-decaffeinate --config ../bulk-decaffeinate.config.js to use a config file one level up in the directory structure.

Many config options can also be specified directly as CLI arguments, with CLI arguments taking precedence over any config file setting.

Here's an example config file:

module.exports = {
  jscodeshiftScripts: [
    './scripts/dev/codemods/arrow-function.js',
    './scripts/dev/codemods/rd-to-create-element.js',
    './scripts/dev/codemods/create-element-to-jsx.js',
  ],
  mochaEnvFilePattern: '^.*-test.js$',
  fixImportsConfig: {
    searchPath: './coffee',
    absoluteImportPaths: ['./coffee'],
  },
};

Specifying files to process

The following config keys can be specified:

• searchDirectory: a path to a directory where bulk-decaffeinate will search for all CoffeeScript files (ignoring files in node_modules directories).
• pathFile: a path to a file containing a list of .coffee file paths to process, one per line.
• filesToProcess: an array of .coffee file paths to process.
• fileFilterFn: a optional JavaScript function that takes the absolute path of a file to consider and returns false if the file should be excluded. This is run after the normal file discovery process, and is useful if there are specific files or directories that should not be converted.

If multiple of searchDirectory, pathFile, or filesToProcess are specified, the union of the files is taken. If none is specified, bulk-decaffeinate will recursively discover all CoffeeScript files in the working directory.

Each of these has a command line arg version, which takes precedence over config file values; see the result of --help for more information.

Common configuration options

• useJSModules: an optional boolean. If true, decaffeinate will be configured to produce code with import/export syntax, and the fix-imports step will be run afterward to correct any import statements across the codebase. The fix-imports step can be configured using fixImportsConfig.
• decaffeinateArgs: an optional array of additional command-line arguments to pass to decaffeinate. For example, ['--keep-commonjs'] sets the preference to keep require and module.exports rather than converting them to import and export.
• jscodeshiftScripts: an optional array of paths to jscodeshift scripts to run after decaffeinate. This is useful to automate any cleanups to convert the output of decaffeinate to code matching your JS style. In addition, you can specify any of the built-in scripts included with this package:
  • prefer-function-declarations.js: change let f = function() {} into function f() {} when possible.
  • remove-coffee-from-imports.js: change import and require statements for .coffee files to no longer specify an extension.
  • top-level-this-to-exports.js: change this at the top level to exports, so the code can run in babel and node.
• fixImportsConfig: an optional object. If present, a whole-codebase pass will be done to fix any incorrect imports involving the converted files. It should be an object with up to two fields:
  • searchPath: a required field specifying a path to a directory containing all JS files in the project.
  • absoluteImportPaths: an optional array of strings, each of which is used as an absolute path starting point when resolving imports. This is necessary if you do any tricks to get absolute-style imports in your project, since the fix-imports script needs to be able to resolve import names to files.

Other configuration

• customNames: an optional object mapping old filename to new filename. By default, the extension is removed and replaced with ".js" (or nothing for extensionless files), but this mapping can be used to override the behavior to provide a specific target directory, name, and/or file extension for any specific files being converted.
• outputFileExtension: an optional file extension, like "ts" or "jsx". If specified, all converted files will have this extension.
• mochaEnvFilePattern: an optional regular expression string. If specified, all generated JavaScript files with a path matching this pattern have the text /* eslint-env mocha */ added to the start. For example, "^.*-test.js$".
• codePrefix: an optional string. If specified, it will be prepended to the start of every affected file, above the autogenerated comment. If the prefix should be on its own line, it should end with a newline.
• landConfig: an object with preferences for the land command. There are three available options:
  • remote: an optional string with the name of the remote component of the branch to base commits off of. Defaults to origin.
  • upstreamBranch: an optional string with the name of the remote branch to base commits off of. Defaults to master. For example, if both remote and upstreamBranch are unspecified, then commits are created based on origin/master.
  • phabricatorAware: an optional boolean that's useful if you're using Phabricator for code review. If specified, the generated commits will all have a proper "Differential Revision" line and the final merge commit will be run through arc amend to pull in the updated commit message.
• landBase: if specified, overrides the auto-detected base commit when running the land command. Generally this is specified on the command line using --land-base rather than in a config file.
• numWorkers: if specified, the number of parallel workers to use for parallel operations like decaffeinate and eslint --fix.
• skipVerify: set to true to skip the initial verification step when running the convert command. This makes bulk-decaffeinate take less time, but if any files fail to convert, it may leave the filesystem in a partially-converted state.
• skipEslintFix: set to true to skip the ESLint step.

Configuring paths to external tools

Rather than having bulk-decaffeinate automatically discover the relevant binaries, you can specify them explicitly. If a path is not specified explicitly, bulk-decaffeinate will first search node_modules, then your PATH, then offer to install the tool globally, so generally it's unnecessary to specify these paths in the config file.

These keys can be specified:

• decaffeinatePath: the path to the decaffeinate binary.
• jscodeshiftPath: the path to the jscodeshift binary.
• eslintPath: the path to the eslint binary.