@bufbuild/buf

What is @bufbuild/buf?

@bufbuild/buf is an npm package that provides tools for working with Protocol Buffers (protobufs). It helps in generating code, linting, and managing protobuf files efficiently. The package is part of the Buf ecosystem, which aims to improve the developer experience when working with protobufs.

What are @bufbuild/buf's main functionalities?

Code Generation

This feature allows you to generate code from your protobuf files. You can specify the input directory containing your .proto files, the output directory for the generated code, and the plugins for the desired languages or frameworks.

const { generate } = require('@bufbuild/buf');

async function generateCode() {
  await generate({
    input: 'path/to/proto/files',
    output: 'path/to/output/directory',
    plugins: ['typescript', 'grpc-web']
  });
}

generateCode();

Linting

Linting helps you ensure that your protobuf files adhere to best practices and style guidelines. This feature scans your .proto files and reports any issues or warnings.

const { lint } = require('@bufbuild/buf');

async function lintProtos() {
  const result = await lint({
    input: 'path/to/proto/files'
  });
  console.log(result);
}

lintProtos();

Breaking Change Detection

This feature allows you to detect breaking changes between different versions of your protobuf files. It compares the current version of your .proto files against a previous version and reports any breaking changes.

const { breaking } = require('@bufbuild/buf');

async function checkBreakingChanges() {
  const result = await breaking({
    against: 'path/to/previous/proto/files',
    input: 'path/to/current/proto/files'
  });
  console.log(result);
}

checkBreakingChanges();

Other packages similar to @bufbuild/buf

protobufjs

protobufjs is a popular library for working with Protocol Buffers in JavaScript. It provides a comprehensive set of tools for parsing, serializing, and generating protobuf messages. Unlike @bufbuild/buf, protobufjs focuses more on runtime operations and less on linting or breaking change detection.

grpc-tools

grpc-tools is a package that provides tools for working with gRPC and Protocol Buffers. It includes a code generator for various languages and a set of utilities for managing gRPC services. While it offers code generation similar to @bufbuild/buf, it does not provide linting or breaking change detection features.

protoc-gen-ts

protoc-gen-ts is a plugin for the Protocol Buffers compiler (protoc) that generates TypeScript code from .proto files. It is specifically focused on TypeScript code generation and does not offer the broader set of features like linting or breaking change detection that @bufbuild/buf provides.

README

Buf

The buf CLI is the best tool for working with Protocol Buffers. It provides:

• A linter that enforces good API design choices and structure.
• A breaking change detector that enforces compatibility at the source code or wire level.
• A generator that invokes your plugins based on configurable templates.
• A formatter that formats your Protobuf files in accordance with industry standards.
• Integration with the Buf Schema Registry, including full dependency management.

Installation

Homebrew

You can install buf using Homebrew (macOS or Linux):

brew install bufbuild/buf/buf

This installs:

• The buf, protoc-gen-buf-breaking, and protoc-gen-buf-lint binaries
• Shell completion scripts for Bash, Fish, Powershell, and zsh

Other methods

For other installation methods, see our official documentation, which covers:

• Installing buf via npm
• Installing buf on Windows
• Using buf as a Docker image
• Installing as a binary, from a tarball, and from source through GitHub Releases
• Verifying releases using a minisign public key

Usage

Buf's help interface provides summaries for commands and flags:

buf --help

For more comprehensive usage information, consult Buf's documentation, especially these guides:

• buf breaking
• buf build
• buf generate
• buf lint
• buf format
• buf registry (for using the BSR)

CLI breaking change policy

We will never make breaking changes within a given major version of the CLI. Once buf reaches v1.0, you can expect no breaking changes until v2.0. But as we have no plans to ever release a v2.0, we will likely never break the buf CLI.

This breaking change policy does not apply to commands behind the buf beta gate, and you should expect breaking changes to commands like buf beta registry. The policy does go into effect, however, when those commands or flags are elevated out of beta.

Our goals for Protobuf

Buf's goal is to replace the current paradigm of API development, centered around REST/JSON, with a schema-driven paradigm. Defining APIs using an IDL provides numerous benefits over REST/JSON, and Protobuf is by far the most stable and widely adopted IDL in the industry. We've chosen to build on this widely trusted foundation rather than creating a new IDL from scratch.

But despite its technical merits, actually using Protobuf has long been more challenging than it needs to be. The Buf CLI and the BSR are the cornerstones of our effort to change that for good and to make Protobuf reliable and easy to use for service owners and clients alike—in other words, to create a modern Protobuf ecosystem.

While we intend to incrementally improve on the buf CLI and the BSR, we're confident that the basic groundwork for such an ecosystem is already in place.

The Buf Schema Registry

The Buf Schema Registry (BSR) is a SaaS platform for managing your Protobuf APIs. It provides a centralized registry and a single source of truth for all of your Protobuf assets, including not just your .proto files but also remote plugins. Although the BSR provides an intuitive browser UI, buf enables you to perform most BSR-related tasks from the command line, such as pushing Protobuf sources to the registry and managing users and repositories.

The BSR is not required to use buf. We've made the core features of the buf CLI available to all Protobuf users.

More advanced CLI features

While buf's core features should cover most use cases, we've included some more advanced features to cover edge cases:

• Automatic file discovery. Buf walks your file tree and builds your .proto files in accordance with your supplied build configuration, which means that you no longer need to manually specify --proto_paths. You can still, however, specify .proto files manually through CLI flags in cases where file discovery needs to be disabled.
• Fine-grained rule configuration for linting and breaking changes. While we do have recommended defaults, you can always select the exact set of rules that your use case requires, with 40 lint rules and 53 breaking change rules available.
• Configurable error formats for CLI output. buf outputs information in file:line:column:message form by default for each lint error and breaking change it encounters, but you can also select JSON and, in the near future, JUnit output.
• Editor integration driven by buf's granular error output. We currently provide linting integrations for both Vim and Visual Studio Code but we plan to support other editors, such as Emacs and JetBrains IDEs like IntelliJ and GoLand, in the future.
• Universal Input targeting. Buf enables you to perform actions like linting and breaking change detection not just against local .proto files but also against a broad range of other Inputs, such as tarballs and ZIP files, remote Git repositories, and pre-built image files.
• Speed. Buf's internal Protobuf compiler compiles your Protobuf sources using all available cores without compromising deterministic output, which is considerably faster than protoc. This allows for near-instantaneous feedback, which is of special importance for features like editor integration.

Next steps

Once you've installed buf, we recommend completing the CLI tutorial, which provides a broad but hands-on overview of the core functionality of the CLI. The tour takes about 10 minutes to complete.

After completing the tour, check out the remainder of the docs for your specific areas of interest.

Community

For help and discussion around Protobuf, best practices, and more, join us on Slack.

For updates on the Buf CLI, follow this repo on GitHub.

For feature requests, bugs, or technical questions, email us at [e-mail]. For general inquiries or inclusion in our upcoming feature betas, email us at [e-mail].

Changelog

[v1.41.0] - 2024-09-11

• Add HTTP/3 support for gRPC with buf curl.
• Fix issue where errors from protoc plugins may be overwritten when executing plugins in parallel.