@sketch-hq/sketch-file-format-marina

sketch-marina-file-format

FlatBuffers Schema for Binary Marina Files.

Getting Started

Requirements

No special tooling is required to work on the schema, simply make your edits to the src/schema.fbs file, and push your changes in a Pull Request.

If your change should result in a new version of the schema being released, then you may need to add a changeset file. Read the Release Process section below for more on this.

Local testing

• Launch nvm use
• Launch yarn
• Install the FlatBuffers compiler.
  • If you're on Mac, you can use homebrew: brew install flatbuffers
• Now you're ready to make the changes to the schema
• To generate the classes needed to parse the schema in swift/cpp/ts, launch: yarn build. The output folder is: dest.

Overview

The file schema.fbs in this repo is a FlatBuffers schema. Using this schema and the flatc tool we can generate type safe APIs in a range of languages for reading and writing binary files that conform to the schema.

In this way we can increase our confidence that different parts of our infrastructure can create and consume compatible binary files, as long as they use APIs generated from the same version of the schema or an evolution of the schema that has followed the rules described in the Compatibility Across Versions section below.

The schema in this repo sits upstream of any pieces of infrastructure that implement it. New schema versions can be released at any time - it will be the responsibility of downstream projects to coordinate standardization on new schema versions.

Compatibility Across Versions

FlatBuffers binaries are designed to facilitate forwards and backwards compatibility. Read the docs for more information but this is achieved by taking into account the following constraints:

• New fields may only added to the end of table definitions
• Ordering of fields in tables should remain stable across versions
• Field data types should not change across versions
• Table fields may be marked deprecated (stop writing them), but never removed from the schema
• Avoiding marking table fields as required

As an illustration, given two schema versions v1 and v2:

	v1 file	v2 file
Reading file using API generated from v1	Fully compatible	- Any new fields present in the v2 file data are ignored by the v1 code - Any fields newly deprecated in v2 and missing from the v2 file data are returned as their default values to the v1 code
Reading file using API generated from v2	- Any new fields defined in v2 and missing from the v1 file data are returned as their default values - Attemping to read deprecated fields is a compile-time error in v2, even if they may be present in the v1 file	Fully compatible

Note: "default values" here means 0 for scalars and some form of null for other types (table and struct references).

Local Builds

A GitHub Action will run the build process for Pull Requests and merges to the main branch, so there's no requirement to be able to run the build locally.

However, if you do decide to build locally, you'll need to install the flat buffers cli, as stated before. Then, you can run:

yarn build

This will generate the APIs in C++, Swift and TypeScript and the tool executables into the dest folder.

Release Process

This repository uses Atlassian Changesets to manage the release process. Read the docs for more information, but the top-level summary is:

• Changesets are YAML files committed to the repo that signal an intent to release a new version of the schema due to a specific change
• A GitHub Action maintains a PR called "Version Packages" that lists the changeset files in the repo, that when manually merged causes the package to be released

Upon release the following actions take place:

• A private npm package is released @sketch-hq/sketch-presentation-file-format
• The build output is commited to a branch named build/vX.X.X where X.X.X is the semver. This allows projects, like Sketch, that manage dependencies via Git submodules to easily access build output via a Git commit
• The repo is tagged with the semver
• A release is added to the repo Releases page
• A Changelog entry is added

Adding a Changeset

You need to add a changeset file to your Pull Request when it contains changes to the schema, or changes that result in different build output. A changeset is not required when making changes to other parts of the repo (e.g. documentation changes etc.).

Add a YAML file to the .changeset folder, with the following contents:

---
"@sketch-hq/sketch-presentation-file-format": <bump-type>
---

A description of the change.

The <bump-type> should be replaced with one of patch, minor or major according to meanings documented in the semver spec.

Alternatively, if you have Yarn installed you may invoke yarn changeset to add the changeset file using a CLI wizard.