veritone-developer-cli

Veritone Developer CLI

⚠️ WARNING: This is under active development and not yet considered stable for production use!

veritone-developer-cli is a command line interface (CLI) for deploying aiWARE applications to an aiWARE instance directly from their source code.

The tool analyzes a specially-formatted manifest.yaml file in the base of a project directory, looks at the difference between the configuration specified there and the live deployment of the application in aiWARE, and calls the aiWARE API to make changes to bring the deployed version of the application inline with the manifest file.

⚠️ WARNING: This means it will CHANGE THINGS IN YOUR ACCOUNT! USE WITH CAUTION!

Usage

1. Install npx

2. In the root directory for your application, create a manifest.yaml that looks like test/manifest.yaml.

3. Retrieve an aiWARE token somehow (probably by logging in and pulling it out of your cookies)

4. Run this to create your application in the production environment:

   veritone-developer-cli deploy
   

   Or for deploying to another environment, you can override the API URL with -u.

   veritone-developer-cli deploy -e "<the URL for your environment (e.g. uk.veritone.com)>"
   

For full usage information, run veritone-developer-cli --help

The Manifest File

The script works against a manifest file, which can be written in either .yaml or .json. You optionally specify the path to the manifest file as the argument to the deploy command.

veritone-developer-cli deploy my-manifest.yaml

If you don't specify, the script will look in the current directory for manifest.yaml or manifest.json (prefers yaml).

TODO: Need to document the format of the actual contents. For now, just look at test/manifest.yaml and copy it.

Commands

For now, there's just one command: deploy. And running npx veritone-developer-cli or ../bin/veritone-developer or ../bin/veritone-developer deploy all do the same thing for the moment.

Environment Variables

To help protect secrets and ease integration into build systems, some command line arguments can also be provided via environment variables. The command line arguments take precedence over environment variables if both are present.

Token	Equivalent Flag
VERI_ENVIRONMENT	--environment
VERI_PASSWORD	--password
VERI_TOKEN	--token
VERI_USERNAME	--username

Developing Locally

Dependencies:

• node.js 8 or higher
• yarn

yarn
yarn link # to mimic a global install so you can use it easily on the command line in other projects

Soruce Code

See the src directory.

Testing

See the test directory.

Publishing New Versions

• Follow semver.
• You'll need to be on the veritone npm team.
• Always publish from master and always commit and tag immediately after publishing.

git checkout master
git pull
yarn publish # and answer prompts.  This will add a commit and tag to your branch.
git push

Roadmap

Inspiration

Feel as magical as now.

Allow developers to do everything Veritone Developer provides without ever having to touch the UI.

Our sketches for the manifest can be seen in our CodeSandbox. See @SteveShaffer for access if you don't have it.

Following Twilio's guide on how to build a node CLI. But using yargs instead of arg.

Known Assumptions and Limitations

• Does NOT present a plan or ask for consent before making API calls.
• Redeploys all engine metadata and publishes a new build every time the deploy script is run.
• Does NOT handle by-reference engines. Assumes the engine is "owned" by the application and defined completely inline.
• Does NOT handle by-reference schemas. Assumes the schema is "owned" by the application and defined completely inline.
• Does NOT de-link schemas if a schema is already linked to the application but it's not in the manifest or was removed from the manifest.
• Does NOT handle auto-generation of IDs for schemas. Schemas are required to have a GUID created by the author.
• Does NOT support breaking changes to schemas. If you manually publish a new breaking version through Developer App and then update the majorVersion number in the manifest, it should be able to push updates to the new major version though.

TODO

• Actually write tests in buildPlan.test-TODO.js
• Move the "Developing Locally" section into a CONTRIBUTING guide and the "Roadmap" section into a ROADMAP document so the main README is more geared toward the public API and npm-friendly.
• Figure out proper name (veritone-developer vs. veri vs. aiware vs. aiware-developer-cli or something like that)
• See TODOs inline in source code
• Get more formal automated tests in place
• Find a better default name of the manifest like .aiware.json
• Have a mode where, instead of interactively calling the API, bundles up an immutable install "package" that can then be uploaded to any environment.
• TypeScript
• Prettier
• Reorganize scripts folder to move all deploy scripts to /deploy and use yargs .commandDir
• Provide a way to "diff" engine build contents to determine if a new build should be pushed. This would probably leverage some sort of checking of hashes or docker image layers or something.

Desired Dependencies

We'd like to be intentional about the dependencies we bring in. There's always lots of technologies to pick from on these projects.

Here's what we'd like to explore using for various things. If there's a better alternative, we might change our minds, but here's where our minds are at right now.

• yargs for parsing CLI args
• pkg for bundling a binary
• listr for progress meters
• chalk for colored STDOUT
• inquirer for interactive inputs
• Integrate into GitHub Actions for CI/CD and publishing to npm

Planned Deploy Targets

• Should have a command line synonym shorter than veritone-developer

• Should be able to run via npx

  npx veritone-developer-cli deploy
  

• Should be able to run as a standalone binary (not exact URL)

  curl https://scripts.veritone.com/veritone-developer | bash