@mapbox/mvt-fixtures - npm Package Compare versions

Comparing version 2.1.0 to 3.0.0-beta1

CHANGELOG.md

@@ -1,7 +0,18 @@
-## 2.1.0 (release candidate 1)
+## 3.0.0-beta2 (next)
 
+- Recreate all fixtures that previously existed in 2.x
+
+## 3.0.0-beta1
+
+- Create source files for all fixtures in the src/ directory - these are node modules which include information about the fixture, including a description, validity specs, and a JSON representation of the fixture.
+- Rely on `pbf` and `protocol-buffers-schema` modules to generate fixtures based on the Mapbox Vector Tile Specification, which is now a submodule to this repository.
+- All fixtures are generated into the fixtures/ directory programmatically using `npm run build`
+- Include a javascript API with `get`, `create`, and `each` methods, described in API.md
+- Automatically generate fixture documentation from the source files, located in FIXTURES.md
+
+## 2.1.0
+
 - Rename project to `mvt-fixtures`
 - Break fixtures into `valid` and `invalid` directories
 - Match version with that of the Mapbox Vector Tile Specification
-- Include more valid fixtures: TODO
 
@@ -14,2 +25,2 @@ ## 1.0.0
 
-- first
+- first

package.json

 {
   "name": "@mapbox/mvt-fixtures",
-  "version": "2.1.0",
+  "version": "3.0.0-beta1",
   "description": "A require-able test fixture suite of valid and invalid Mapbox Vector Tiles",
-  "main": "./lib/index.js",
+  "main": "index.js",
   "scripts": {
-    "test": "tape test/*.test.js"
+    "test": "tape test/*.test.js",
+    "docs": "node scripts/docs.js && documentation build index.js -f md > API.md --shallow",
+    "new": "node scripts/new.js",
+    "build": "node scripts/build.js"
   },
@@ -17,4 +20,10 @@ "homepage": "https://github.com/mapbox/mvt-fixtures",
   "devDependencies": {
+    "@mapbox/vector-tile": "^1.3.0",
     "tape": "^4.5.1"
+  },
+  "dependencies": {
+    "d3-queue": "^3.0.7",
+    "pbf": "^3.0.5",
+    "protocol-buffers-schema": "^3.3.1"
   }
-}
+}

README.md

 # mvt-fixtures
 
-![](https://travis-ci.org/mapbox/evilmvt.svg?branch=master)
+[![Build Status](https://travis-ci.org/mapbox/mvt-fixtures.svg?branch=master)](https://travis-ci.org/mapbox/mvt-fixtures)
 
-A [`require()`able](#require-fixtures) suite of [valid](#valid-fixtures) & [invalid](#invalid-fixtures) vector tile fixtures for testing [Mapbox Vector Tile](https://github.com/mapbox/vector-tile-spec) decoding. *Previously called `evilmvt` but eventually `happymvt` prevailed.*
+A `require()`able suite of valid and invalid vector tile fixtures for testing [Mapbox Vector Tile](https://github.com/mapbox/vector-tile-spec) encoders and decoders. You can view a list of all fixtures at [FIXTURES.md](FIXTURES.md).
 
-All fixtures are included in the `/fixtures` directory. They are named to be as descriptive as possible, but the tables below gives us more space to describe the underlying data.
+# Usage
 
-**Version**: The version of `mvt-fixtures` stays in sync with the version of the Mapbox Vector Tile Specification. For instance `mvt-fixtures@2.1.0` references the Mapbox `vector-tile-spec@2.1`, reserving the patch versions for any unexpected bug fixes in this project.
+mvt-fixtures can be used in two distinct ways
 
-# Usage
+1. **javascript interface**: use the javascript interface to generate fixtures on the fly
+1. **raw fixtures** use the raw fixtures directly via the /fixtures directory.
 
-### `require('@mapbox/mvt-fixtures')`
+The Javascript API is recommended if you are working in Javascript or Node.js. The raw fixtures are provided for those using this outside of a Javascript application. The recommended workflow is to have your encoder or decoder loop through every fixture and either expect to successfully decode/encode valid fixtures, or fail to decode/encode invalid fixtures. When new fixtures are added to this repository, you simply need to update the version of the module (or your submodule) to get the new fixtures and re-run tests.
 
-Install
+**Validity:** each fixture includes information about whether they are valid according to the specification versions and possible error outcomes if they are invalid. If any of the fixtures are invalid, they must include an `error` field describing how to recover (or not) from the error. These can be found in the `validity` field of the fixture and info.json files. The following checks:
 
+* `v1` (Boolean): is this fixture valid according to Version 1.x of the Mapbox Vector Tile spec
+* `v2` (Boolean): is this fixture valid according to Version 2.x of the Mapbox Vector Tile spec
+* `error` (String): describes if the encoder/decoder should recover from this error or stop completely. THis is only present if the fixture is invalid according to one or more spec revisions. Values are
+  * `recoverable`: should the encoder/decoder continue move on and continue its work? For instance, if invalid geometry is found, can the encoder safely move to the next feature?
+  * `fatal`: the encoder should completely stop its process
+
+### Javascript usage
+
+Check out the full Javascript interface over at [API.md](API.md)
+
+```shell
+npm install @mapbox/mvt-fixtures --save-dev
 ```
-npm install @mapbox/mvt-fixtures --save
+
+```javascript
+const mvtf = require('@mapbox/mvt-fixtures');
+const decoder = require('your-mvt-decoder');
+
+mvtf.each(function(fixture) {
+  let output = decoder(fixture.buffer);
+  assert.equal(output.layers.length, fixture.json.layers.length, 'expected number of layers');
+  // ... more tests
+});
 ```
 
-You can require the fixtures directly from the `evilmvt` module using the name of the fixture.
+### Non-JS interface
 
+You can access all of the fixtures and their metadata in the /fixtures directory. You can download this repository and get them manually, or use this repository as a submodule. Each fixture is named by the directory /fixtures/{name} and has the following files:
+
+1. tile.mvt - the protocol buffer that represents (or intentionally breaks) the Mapbox Vector Tile specification
+1. tile.json - a JSON representation of the tile and its properties
+1. info.json - information about the fixture including `name`, `description`, and `specification_reference`.
+
+# Develop
+
+### Adding a new fixture
+
+All fixtures have a source file in the /src directory. This file is a module that exports an object with the following parameters:
+
 ```javascript
-var fixtures = require('@mapbox/mvt-fixtures').fixtures;
-var buffer = fixtures.invalid['Tags-nonexistant-values'];
-// do something with bufer
+module.exports = function(schema) {
+  return {
+    description: 'DESCRIPTION',
+    specification_reference: 'SPECIFICATION_URL',
+    validity: {
+      v1: true,
+      v2: false,
+      error: 'ERROR_TYPE'
+    },
+    json: {...},
+    manipulate: function(buffer) {
+      // function to further manipulate the buffer
+      return buffer;
+    }
+  }
+};
 ```
 
-### Vendor or Submodule
+A new fixture can be created by running the command, which will auto-increment the ID:
 
-Alternatively, you can [download the repository](https://github.com/mapbox/evilmvt/archive/master.zip) and use the fixtures manually, OR include this repository as a [Git Submodule](https://github.com/blog/2104-working-with-submodules).
+```shell
+npm run new
+# New file created: /src/003.js.
+```
 
-# Invalid fixtures `fixtures/invalid`
+### Building fixtures
 
-Fixture name | Description
----|---
-`Feature-missing-GeomType.mvt` | The `Feature` message is missing a [`GeomType`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L41) message.
-`Feature-multiple-geometries.mvt` | The `Feature` message as multiple [`geometry`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L46) fields encoded, when there should only be one.
-`Feature-no-geometry.mvt` | The `Feature` message has no [`geometry`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L46).
-`Feature-odd_number_tags.mvt` | Only has a single [`tag`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L38) where multiples of 2 are required.
-`Feature-unknown_field_type.mvt` | Has a field value of `10`, which is [not listed as an enum](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L8-L13) and therefore invalid.
-`GeomType-type.mvt` | The tag for GeomType is `10`, which is invalid.
-`Key-mistyped_uint32.mvt` | Has a [`key`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L63) property incorrectly encoded as a type `std::uint32_t`. | n/a 
-`Layer-extent-mistyped_string.mvt` | Layer [`extent`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L70) is incorrectly encoded as a type `std::string`.
-`Layer-extent-none.mvt` | Missing the [`extent`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L70) type
-`Layer-name-duplicates.mvt` | Includes two layer [`name`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L57)s with the same value: "layer_name".
-`Layer-name-mistyped_uint32.mvt` | Has a layer name incorrectly encoded as `std::uint32_t`.
-`Layer-name-none.mvt` | Does not include a layer name.
-`Layer-name-none-version1.mvt` | Same as above, but version 1 tile.
-`Layer-no-features.mvt` | Layer has no repeated [`Features`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L60) tags.
-`Layer-unknow_value_type.mvt` | Includes a Layer value tag of `20`, which is not defined in the spec.
-`Layer-invalid-version.mvt` | Layer version is `99`, which is invalid according to the specification.
-`Layer-version-mistyped_string.mvt` | Layer version is incorrectly typed as a `std::string`.
-`Layer-version-none.mvt` | Layer does not have a [`version`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L55) property.
-`Tags-nonexistant-values.mvt` | Feature has [`tags`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L38) that point to non-existent [`Keys`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L63) and [`Values`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) in the layer.
-`Tile-unknown-tag.mvt` | Tile message has an unknown tag value. The only accepted tag value here is `3`, but this tile encodes a `Feature` with the tag value of `10`.
-`Value-no-fields.mvt` | includes a [`Value`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) without any fields encoded within it.
-`Value-multiple-fields.mvt` | The [`Value`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) message has two entries, both strings, where there should only be one.
-`Value-string-mistyped_int64.mvt` | A Layer value property is listed as "string" but encoded as `std::int64_t`.
-`Value-unknown-field-type.mvt` | The [`Value`](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) has a field with an [unknown type](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L17-L28).
+To rebuild all of the raw fixtures (including the tile.mvt, tile.json, and info.json files) in /fixtures you can run:
 
-# Valid fixtures `fixtures/valid`
+```
+npm run build
+```
 
-Fixture name | Description
----|---
-`Feature-single-linestring.mvt` | Single layer with a valid [linestring geometry](https://github.com/mapbox/vector-tile-spec/tree/master/2.1#4353-example-linestring) from the spec docs.
-`Feature-single-multilinestring.mvt` | Single layer with a valid [multilinestring geometry](https://github.com/mapbox/vector-tile-spec/tree/master/2.1#4354-example-multi-linestring) from the spec docs.
-`Feature-single-multipoint.mvt` | Single layer with a valid [multipoint geometry](https://github.com/mapbox/vector-tile-spec/tree/master/2.1#4352-example-multi-point) from the spec docs.
-`Feature-single-point.mvt` | Single layer with a valid [point geometry](https://github.com/mapbox/vector-tile-spec/tree/master/2.1#4351-example-point) from the spec docs.
-`Feature-single-polygon.mvt` | Single layer with a valid [polygon geometry](https://github.com/mapbox/vector-tile-spec/tree/master/2.1#4355-example-polygon) from the spec docs.
-`Feature-unknown-GeomType.mvt` | Single geometry with [`UNKNOWN` type](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L9). This is considered "valid" in the lens of the specification. Encoders/decoders can choose to use or throw on this goemetry type.
-`Value-single-bool-point.mvt` | Single [Value](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) with `bool` type and a single Point feature.
-`Value-single-double-point.mvt` | Single [Value](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) with `double` type and a single Point feature.
-`Value-single-float-point.mvt` | Single [Value](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) with `float` type and a single Point feature.
-`Value-single-int64-point.mvt` | Single [Value](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) with `int64` type and a single Point feature.
-`Value-single-sint64-point.mvt` | Single [Value](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) with `sint64` type and a single Point feature.
-`Value-single-string-point.mvt` | Single [Value](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) with `string` type and a single Point feature.
-`Value-single-uint64-point.mvt` | Single [Value](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L66) with `uint64` type and a single Point feature.
-`Values-all.mvt` | A buffer with all possible [`Value` types](https://github.com/mapbox/vector-tile-spec/blob/master/2.1/vector_tile.proto#L17-L28) encoded in the layer and single Feature.
+### Building docs
+
+Documentation takes two forms...
+
+1. Javascript API docs in API.md
+1. Fixture reference in FIXTURES.md
+
+These can be generated by running:
+
+```
+npm run docs
+```
+
+### Running tests
+
+All tests can be run with:
+
+```
+npm test
+```

test/fixtures.test.js

@@ -1,16 +0,49 @@
-var test = require('tape');
-var fixtures = require('../lib/').fixtures;
+'use strict';
 
-test('valid fixtures are buffers', function(t) {
-  for (f in fixtures.valid) {
-    t.ok(Buffer.isBuffer(fixtures.valid[f]), f + ' is a buffer');
-  }
-  t.end();
+const test = require('tape');
+const fs = require('fs');
+const path = require('path');
+const mvtf = require('..');
+
+test('[fixtures] validate all raw fixtures are present', (assert) => {
+  mvtf.each(function(f) {
+    assert.ok(fs.existsSync(path.resolve(`${__dirname}/../fixtures/${f.id}`)), 'directory exists');
+    assert.ok(fs.existsSync(path.resolve(`${__dirname}/../fixtures/${f.id}/tile.mvt`)));
+    assert.ok(fs.existsSync(path.resolve(`${__dirname}/../fixtures/${f.id}/tile.json`)));
+    assert.ok(fs.existsSync(path.resolve(`${__dirname}/../fixtures/${f.id}/info.json`)));
+  });
+  assert.end();
 });
 
-test('invalid fixtures are buffers', function(t) {
-  for (f in fixtures.invalid) {
-    t.ok(Buffer.isBuffer(fixtures.invalid[f]), f + ' is a buffer');
-  }
-  t.end();
-});
+test('[fixtures] validate all raw fixtures info matches that of the source fixture', (assert) => {
+  mvtf.each(function(f) {
+    // read info file and check name, description, and spec url match
+    let info = JSON.parse(fs.readFileSync(path.resolve(`${__dirname}/../fixtures/${f.id}/info.json`)));
+    assert.equal(info.description, f.description, 'descriptions match');
+    assert.equal(info.specification_reference, f.specification_reference, 'specification_references match');
+
+    let buffer = fs.readFileSync(path.resolve(`${__dirname}/../fixtures/${f.id}/tile.mvt`));
+    assert.deepEqual(buffer, f.buffer, 'buffers are equal');
+    assert.equal(buffer.length, f.buffer.length, 'buffer lengths are equal');
+
+    let json = JSON.parse(fs.readFileSync(path.resolve(`${__dirname}/../fixtures/${f.id}/tile.json`)));
+    assert.deepEqual(json, f.json, 'jsons are equal');
+  });
+
+  assert.end();
+});
+
+test('[fixtures] validate all source fixtures to make sure they have all required properties', (assert) => {
+  const files = fs.readdirSync(path.resolve(`${__dirname}/../src`));
+  files.forEach(function(file) {
+    let fixture = require(path.resolve(`src/${file}`))(mvtf.schema);
+    assert.ok(fixture.description, `${file} has property description`);
+    assert.ok(fixture.specification_reference, `${file} has property specification_reference`);
+    assert.ok(fixture.json, `${file} has property json`);
+    if (fixture.manipulate) {
+      assert.equal(typeof fixture.manipulate, 'function', `${file} property manipulate is a function`);
+    }
+  });
+
+  assert.end();
+});

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Major refactor

Supply chain risk

Package has recently undergone a major refactor. It may be unstable or indicate significant internal changes. Use caution when updating to versions that include significant changes.

Found 1 instance in 1 package

Shell access

Supply chain risk

This module accesses the system shell. Accessing the system shell increases the risk of executing arbitrary code.

Found 1 instance in 1 package

Dynamic require

Supply chain risk

Dynamic require can indicate the package is performing dangerous or unsafe dynamic code execution.

Found 1 instance in 1 package

No v1

Quality

Package is not semver >=1. This means it is not stable and does not support ^ ranges.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Native code

Supply chain risk

Contains native code (e.g., compiled binaries or shared libraries). Including native code can obscure malicious behavior.

Found 37 instances in 1 package

Improved metrics

Total package byte prevSize

101324

increased by702.19%

Lines of code

541

increased by1765.52%

Number of lines in readme file

111

increased by40.51%

Number of medium supply chain risk alerts

2

decreased by-94.59%

Worsened metrics

Dependency count

3

increased byInfinity%

Dev dependency count

2

increased by100%

Number of package files

40

decreased by-14.89%

Number of low quality alerts

1

increased byInfinity%

Number of low supply chain risk alerts

10

increased by900%

Dependency changes

• + Addedd3-queue@^3.0.7

• + Addedpbf@^3.0.5

• + Addedprotocol-buffers-schema@^3.3.1

• + Addedd3-queue@3.0.7(transitive)

• + Addedieee754@1.2.1(transitive)

• + Addedpbf@3.3.0(transitive)

• + Addedprotocol-buffers-schema@3.6.0(transitive)

• + Addedresolve-protobuf-schema@2.1.0(transitive)