veritone-developer-cli

Veritone Developer CLI

⚠️ WARNING: This is under active development and not yet considered stable for production use!

veritone-developer-cli is a command line interface (CLI) for deploying aiWARE applications to an aiWARE instance directly from their source code.

The tool analyzes a specially-formatted manifest.yaml file in the base of a project directory, looks at the difference between the configuration specified there and the live deployment of the application in aiWARE, and calls the aiWARE API to make changes to bring the deployed version of the application inline with the manifest file.

⚠️ WARNING: This means it will CHANGE THINGS IN YOUR ACCOUNT! USE WITH CAUTION!

Usage

1. Install npx

2. In the root directory for your application, create a manifest.yaml that looks like test/manifest.yaml.

3. Retrieve an aiWARE token somehow (probably by logging in and pulling it out of your cookies)

4. Run this to create your application in the production environment:

   veritone-developer-cli deploy
   

   Or for deploying to another environment, you can override the API URL with -u.

   veritone-developer-cli deploy -e "<the URL for your environment (e.g. uk.veritone.com)>"
   

For full usage information, run veritone-developer-cli --help

The Manifest File

The script works against a manifest file, which can be written in either .yaml or .json. You optionally specify the path to the manifest file as the argument to the deploy command.

veritone-developer-cli deploy my-manifest.yaml

If you don't specify, the script will look in the current directory for manifest.yaml or manifest.json (prefers yaml).

TODO: Need to document the format of the actual contents. For now, just look at test/manifest.yaml and copy it.

Commands

For now, there's just one command: deploy. And running npx veritone-developer-cli or ../bin/veritone-developer or ../bin/veritone-developer deploy all do the same thing for the moment.

Environment Variables

To help protect secrets and ease integration into build systems, some command line arguments can also be provided via environment variables. The command line arguments take precedence over environment variables if both are present.

Token	Equivalent Flag
VERI_ENVIRONMENT	--environment
VERI_PASSWORD	--password
VERI_TOKEN	--token
VERI_USERNAME	--username

Developing Locally

Dependencies:

• node.js 8 or higher
• yarn

yarn
yarn link # to mimic a global install so you can use it easily on the command line in other projects

Testing

Running the following should successfully register a full-featured application.

⚠️ WARNING: This DOES make changes in your account. Use with caution!

cd test
VERI_TOKEN="<your token>" ../bin/veritone-developer # or just veritone-developer if you yarn link'd above

Running it again should ideally make no changes.

Publishing New Versions

• Follow semver.
• You'll need to be on the veritone npm team.
• Always publish from master and always commit and tag immediately after publishing.

git checkout master
git pull
yarn publish # and answer prompts.  This will add a commit and tag to your branch.
git push

Roadmap

Inspiration

Feel as magical as now.

Allow developers to do everything Veritone Developer provides without ever having to touch the UI.

Our sketches for the manifest can be seen in our CodeSandbox. See @SteveShaffer for access if you don't have it.

Following Twilio's guide on how to build a node CLI. But using yargs instead of arg.

Known Assumptions and Limitations

• The manifest structure is not validated and is assumed to be "well-formed"
• Does NOT present a plan or ask for consent before making API calls.
• Redeploys all engine metadata and publishes a new build every time the deploy script is run.
• Does NOT handle by-reference engines. Assumes the engine is "owned" by the application and defined completely inline.
• Does NOT handle by-reference schemas. Assumes the schema is "owned" by the application and defined completely inline.
• Does NOT de-link schemas if a schema is already linked to the application but it's not in the manifest or was removed from the manifest.
• Does NOT handle auto-generation of IDs for schemas. Schemas are required to have a GUID created by the author.
• Does NOT support breaking changes to schemas. If you manually publish a new breaking version through Developer App and then update the majorVersion number in the manifest, it should be able to push updates to the new major version though.

TODO

• Figure out proper name (veritone-developer vs. veri vs. aiware vs. aiware-developer-cli or something like that)
• Add option for dry run (i.e. don't perform any GraphQL mutations, just log what WOULD HAVE been performed)
• See TODOs inline in source code
• Get more formal automated tests in place
• Find a better default name of the manifest like .aiware.json
• Tell the user what's going to happen and gather consent (like tf plan)
• Have a mode where, instead of interactively calling the API, bundles up an immutable install "package" that can then be uploaded to any environment.
• TypeScript
• Prettier
• Incorporate interactive login. Ideally, only ask the user for their username and password the first time. And only use it to coin an API key that you store in some config file for use in the future. But there will probably be blockers to that at our API level.
• Reorganize scripts folder to move all deploy scripts to /deploy and use yargs .commandDir
• Provide a way to "diff" engine build contents to determine if a new build should be pushed. This would probably leverage some sort of checking of hashes or docker image layers or something.

Desired Dependencies

We'd like to be intentional about the dependencies we bring in. There's always lots of technologies to pick from on these projects.

Here's what we'd like to explore using for various things. If there's a better alternative, we might change our minds, but here's where our minds are at right now.

• yargs for parsing CLI args
• pkg for bundling a binary
• listr for progress meters
• chalk for colored STDOUT
• inquirer for interactive inputs
• Integrate into Veritone Jenkins environment for CI/CD and publishing to npm

Planned Deploy Targets

• Should have a command line synonym shorter than veritone-developer

• Should be able to run via npx

  npx veritone-developer-cli deploy
  

• Should be able to run as a standalone binary

  curl https://scripts.veritone.com/veritone-developer | bash