datacatalog-custom-model-manager

datacatalog-custom-model-manager

A Python package intended to load user-specified metadata models into Google Cloud Data Catalog, comprising Custom Entries, Tag Templates, and Tags.

It is powered by datacatalog-custom-entries- manager, datacatalog- tag-template-processor, and datacatalog-tag-manager, leveraging their features in the format of a single CLI.

Load Custom Model

From CSV files

• SCHEMAS

Loading custom models from CSV requires a folder (--files-folder argument in the below commands) containing three files: entries.csv, tag_templates.csv, and tags.csv. Please refer to the following external resources for further details on their layouts:

FILE	LAYOUT DOCUMENTATION
entries.csv	datacatalog-custom-entries-manager README
tag_templates.csv	datacatalog-tag-template-processor README
tags.csv	datacatalog-tag-manager README

• SAMPLE INPUT

• sample-input/egeria-business-glossary for reference;
• Sample metadata for datacatalog-custom-model-manager (Google Sheets) might help to create/export the mandatory CSV files.

• COMMANDS

Python + virtualenv

pip install datacatalog-custom-model-manager

datacatalog-custom-model load \
  --files-folder <CSV-FILES-PATH> \
  --project-id <YOUR-PROJECT-ID> --location-id <YOUR-LOCATION-ID> \
  [--delete-existing-tag-templates]

Docker

docker build --rm --tag datacatalog-custom-model-manager .

docker run --rm --tty \
  --volume <CREDENTIALS-FILE-FOLDER>:/credentials --volume <CSV-FILE-FOLDER>:/data \
  datacatalog-custom-model-manager load \
  --files-folder <CSV-FILES-PATH> \
  --project-id <YOUR-PROJECT-ID> --location-id <YOUR-LOCATION-ID> \
  [--delete-existing-tag-templates]