This tool is for rapid universal extension and universal template project creation. This tool is written on Python and can work on Windows or Linux platforms.
All the configuration is located in .uip_config.yml file located on the script folder or home folder of the user or you can use a configuration by using --config option.
Install quip from pip
pip install --upgrade quip-cli
Check the version
quip --version
Check configuration
quip config
Set the path of the baseline projects to source: and template_source: fields in .uip_config.yml file. Folder paths must be full path not relative paths.
Update the fields in the .uip_config.yml file
defaults: # Default values
template: ue-task # default template for uip init command
bootstrap:
source: /projects/dev/ue/ue-baseline # path of extension baseline project
template_source: /projects/dev/ut/ut-baseline # path of universal template baseline project
exclude: # folders that will be excluded for extension
- .git
- .uip
template-exclude: # folders that will be excluded for universal template
- .git
icon_font: cour.ttf # Font for the default icon
use_keyring: true # Keep the passwords in OS level Keyring [Recommended]
project_prefix: cs # This keyword will be added to the folder name and repositories
external:
gitlab:
url: https://gitlab.stonebranch.com
ssl_verify: false
username: <username>
default_group: cs-uac # default group for new repositories
groups: # List of repositories that can be used for new repositories
- integration-prototypes
- cs-uac
protected_branch: # information about protected branch
name: main
merge: DEVELOPER # possible options are:
push: NO_ACCESS # DEVELOPER, MAINTAINER, ADMIN, GUEST, NO_ACCESS, OWNER, REPORTER
default_branch: develop # Name of the default branch
git-init: true # it will run some git commands to initialize default branch
jenkins:
url: https://jenkins.stonebranch.com/
ssl_verify: false
username: <username>
default_group: cs-uac
groups: # Group mapping between GitLab and Jenkins
integration-prototypes: integration-prototypes
cs-uac: cs-uac
sonarqube:
url: https://sonarqube.stonebranch.com
username: <username>
use_token: true # Use Username/Password or Token
ssl_verify: false
groups: # Group mapping between Jenkins and SonarQube
cs-uac: cs-uac
extension.yml: # default values for extension.yml
extension:
name: ""
version: "1.0.0"
api_level: "1.2.0"
requires_python: ">=3.7"
owner:
name: Developer Name
organization: Stonebranch Inc.
comments: Created using ue-task template
uip.yml: # dafault values for .uip/config/uip.yml
userid: ops.admin
url: http://localhost:8090/uc
quip --help
======= QUIP (v.1.7.0-BETA) =======
usage: quip [-h] [--version] [--config CONFIG] [--debug] {new,update,u,up,fields,f,fi,delete,d,del,clone,c,cl,copy,bootstrap,bs,boot,bst,baseline,upload,push,download,pull,build,b,dist,zip,icon,resize-icon,ri,resize,clean,clear,setup,version,config} ...
Wrapper for UIP command.
positional arguments:
{new,update,u,up,fields,f,fi,delete,d,del,clone,c,cl,copy,bootstrap,bs,boot,bst,baseline,upload,push,download,pull,build,b,dist,zip,icon,resize-icon,ri,resize,clean,clear,setup,version,config}
new Creates new integration
update (u, up) Updates existing integration
fields (f, fi) Updates or dumps template.json fields.
delete (d, del) Deletes the integration folder
clone (c, cl, copy)
Clones existing integration with a new name
bootstrap (bs, boot, bst, baseline)
Bootstrap new integration from baseline project
upload (push) Uploads the template to Universal Controller. (Template Only)
download (pull) Download the template from Universal Controller. (Template Only)
build (b, dist, zip)
Builds a zip file to import to Universal Controller. (Template Only)
icon (resize-icon, ri, resize)
Resize the images to 48x48 in src/templates/
clean (clear) Clears the dist folders
setup Setup External Systems
version shows the version of the template/extension
config show the configuration
options:
-h, --help show this help message and exit
--version show program's version number and exit
--config CONFIG, -c CONFIG
path of the global config. Default is ~/.uip_config.yml
--debug, -v show debug logs
quip new ue-new-extension
This command will
quip bootstrap ue-bs-extension
This command will
src folder from the source projectquip clone ue-cloned-extension ..\ue\ue-aws-m2
quip update ue-cloned-extension
To update the SysID values add --update-uuid (or -u) parameter
quip update ue-cloned-extension --uuid
To update the new_uuid values add --new-uuid (or -n) parameter. For any new object you can use a keyword like "new_uuid" as value and it will be automatically updated with this command.
quip update ue-cloned-extension --new-uuid
quip delete ue-cloned-extension
This command will find all the image files under src/templates/ folder (it will ignore template_icon.png and any file that ends with "_48x48.png") and convert them to 48x48 size PNG files. quip will create new files with name *_48x48.png for each image file. If there is only one image file than it will also update the template_icon.png file.
quip icon
If you don't have any image file, you can also create one by using --generate parameter.
quip new ue-new-extension
This command will
quip bootstrap ut-bs-extension --template
This command will
src folder from the source projectquip clone ut-cloned-extension ..\ue\ut-aws-m2 --template
This command will download the template.json, script files and ICON of the template.
This command will
quip download Snowflake --template
This command will not update the Universal Template and the ICON.
quip upload Snowflake --template
quip update ut-cloned-extension
To update the SysID values add --update-uuid (or -u) parameter
quip update ut-cloned-extension --uuid
To update the new_uuid values add --new-uuid (or -n) parameter. For any new object you can use a keyword like "new_uuid" as value and it will be automatically updated with this command.
quip update ut-cloned-extension --new-uuid
quip delete ut-cloned-extension
This command will create a zip file for universal templates. This command will not work for universal extensions because uip build command already has this function.
The zip file will be created under build folder.
quip build ut-databricks -t
This command will find all the image files under src/templates/ folder (it will ignore template_icon.png and any file that ends with "_48x48.png") and convert them to 48x48 size PNG files. quip will create new files with name *_48x48.png for each image file. If there is only one image file than it will also update the template_icon.png file.
quip icon
If you don't have any image file, you can also create one by using --generate parameter.
You can use --config option to select the config file you want to use. This option will allow to use different controllers while downloading or uploading or using different baseline projects based on the project you will create. For example in the following example, quip will download Snowflake universal template from Atlanta Controller.
quip download Snowflake --template --config ~/.uip_config_atlanta.yml
This command will look for multiple files and will try to find the version information. It will show the versions in the files and also will display the possible new versions.
quip version
You can update the version by using 3 different update level: minor, major or release. Minor will update the last part of the version, major will update the middle value and release will update the first value. For example if the current version is 5.22.0, these are the possible next versions;
RELEASE: 6.0.0
MAJOR: 5.23.0
MINOR: 5.22.1
To update the version you can use this command.
quip version minor
if you want to change the version to a specific value, you can use the --force option. For example the following command will change the version to 5.22.4
quip version --force 5.22.4
If you have different version in the different files, in this case you can also use --force option to update them to a specific version. If there are multiple versions you can not use minor, major or release options.
By default quip will check the following files.
You can change this configuration in your .uip_config.yml file. Here is a sample configuration. Quip will use the default configuration if you don't have any configuration in the config.
defaults: # Default values
template: ue-task # default template for uip init command
bootstrap:
source: /Users/HuseyinGomleksizoglu/SE_Projects/dev/ue/ue-baseline # path of extension baseline project
template_source: /Users/HuseyinGomleksizoglu/SE_Projects/dev/ut/ut-baseline # path of universal template baseline project
exclude: # folders that will be excluded for extension
- .git
- .uip
template-exclude: # folders that will be excluded for universal template
- .git
extension.yml: # default values for extension.yml
extension:
name: ""
version: "1.0.0"
api_level: "1.2.0"
requires_python: ">=3.7"
owner:
name: Huseyin Gomleksizoglu
organization: Stonebranch Inc.
comments: Created using ue-task template
uip.yml: # dafault values for .uip/config/uip.yml
userid: ops.admin
url: http://localhost:8090/uc
version_files:
- file: "src/extension.yml"
format: "yml"
location: "extension.version"
- file: "script.yml"
format: "yml"
location: "script.version"
- file: "src/extension.py"
format: "regex"
location: "^\\s*(gl_version|version|__version__)\\s*=\\s*[\\\"']+([^\\\"']+)[\\\"']+"
group: 2
- file: "src/__init__.py"
format: "regex"
location: "^\\s*(gl_version|version|__version__)\\s*=\\s*[\\\"']+([^\\\"']+)[\\\"']+"
group: 2
- file: "src/templates/script*"
format: "regex"
location: "^\\s*(gl_version|version|__version__)\\s*=\\s*[\\\"']+([^\\\"']+)[\\\"']+"
group: 2
- file: "quip/__init__.py"
format: "regex"
location: "^\\s*(gl_version|version|__version__)\\s*=\\s*[\\\"']+([^\\\"']+)[\\\"']+"
group: 2
- file: src/templates/template.json
format: "regex"
location: "\"description\"\\s*:\\s*\"v([\\d\\.]+)"
group: 1
You can also use different configuration files if needed.
quip -c ~/.uip_config_sample.yml version
quip will automatically create a file called fields.yml on the root of the project folder. This will will have yaml representation of the fields. You can modify this file and update the template.json file with the new values.
This command will
quip fields --update
This command will convert the fields information from template.json file and create/update fields.yml file.
quip fields --dump
This command will print some sample code to assign the fields or define dynamic choice fields.
quip fields --update --code
Output:
self.action = fields.get("action", [None])[0]
self.credentials = { "user": fields.get("credentials.user", None), "password": fields.get("credentials.password", None) }
self.end_point = fields.get("end_point", None)
self.region = fields.get("region", None)
self.application = fields.get("application", [None])[0]
self.batch_format = fields.get("batch_format", [None])[0]
self.jcl_file_name = fields.get("jcl_file_name", None)
self.jcl_file_path = fields.get("jcl_file_path", None)
self.script_name = fields.get("script_name", None)
self.jcl_file_name_temp = fields.get("jcl_file_name_temp", None)
self.application_id = fields.get("application_id", None)
self.batch_execution_id = fields.get("batch_execution_id", None)
self.step_name = fields.get("step_name", None)
self.procstep_name = fields.get("procstep_name", None)
self.templib = fields.get("templib", None)
self.parameters = fields.get("parameters", None)
@dynamic_choice_command("application")
def get_application(self, fields):
_fields = []
return ExtensionResult(
rc = 0,
message = "Available Fields: '{}'".format(_fields),
values = _fields
)
A simple file will look like this.
name: AWS Mainframe Modernization
template_type: Extension
agent_type: Any
description: AWS M2 - Mainframe Modernization
extension: aws-m2
min_release: 7.0.0.0
var_prefix: ext
sys_id: fb7fc3a05eac4d57a74fe1b1342b3938
common_script: false
always_cancel: true
send_environment: Launch, Dynamic Choice
send_variables: Local
fields:
- action: Choice
hint: Select the action you want to run
start: true
field_mapping: Choice Field 1
items:
- list-environments
- list-applications
- start-batch
- sync-start-batch
- credentials: Credential
hint: Put Access key to Runtime User and Secret Key to Password fields
label: AWS Credentials
end: true
field_mapping: Credential Field 1
allow_variable: true
span: 2
- end_point: Text
hint: This field is optional. If you don't put the value it will be generated by
using the region value.
start: true
default: https://m2.us-east-1.amazonaws.com
field_mapping: Text Field 4
regex: http.*
- application: Choice
hint: Application will be retrieved from AWS automatically
span: 2
dynamic: true
field_mapping: Choice Field 2
items: []
dependencies:
- end_point
- region
- credentials
show_if:
action: start-batch,sync-start-batch
required: true
- application_id: Text
start: true
restriction: Output Only
field_mapping: Text Field 1
show_if:
action: start-batch,sync-start-batch
- parameters: Array
start: true
name_title: Parameter Name
value_title: Parameter Value
field_mapping: Array Field 1
span: 2
values:
- task_name
- template: Value
- variable_1: Value 1
- variable_2: Value 2
Fields of Universal Template definitions:
Format of the child elements are like this.
First value will be the name of the field and the value of that element will be the type of the field.
Field Types can be one of these items
Other Fields:
titles: Field Name, Field ValueThis field is required for Choice type. This field will include the items of the choice field. Items can be a simple string without a space and it will be used as the name of the item and label will be generated from the names. If you want to give a specific label, in this case set it like this. - use_ssl: Use SSL
- action: Choice
hint: Select the action you want to run
start: true
field_mapping: Choice Field 1
items:
- list-environments
- list-applications
- start-batch
- sync-start-batch: Start Batch and Wait
This field is only for dynamic choice fields. This field will include the list of dependencies of the the dynamic choice field. Dependencies will be the list of field names.
- application: Choice
hint: Application will be retrieved from AWS automatically
span: 2
dynamic: true
field_mapping: Choice Field 2
items: []
dependencies:
- end_point
- region
- credentials
If the field will be displayed based on another field, in this case you can use the show_if option. First element will be name of the dependent field and value of the element will be the condition. If the field will be required if it is visible, than required: true option must be added.
The following example will show the application_id field if the action field is start-batch or sync-start-batch
- application_id: Text
start: true
restriction: Output Only
field_mapping: Text Field 1
show_if:
action: start-batch,sync-start-batch
Other options:
If the field will be required based on another field, in this case you can use the require_if option. First element will be name of the dependent field and value of the element will be the condition.
- task_name: Text
start: true
field_mapping: Text Field 1
require_if:
action: start-batch,sync-start-batch
Some of the options not implemented in quip because they are not common options. For these options you can use the raw: option. This item will include the list of items and values of them.
- backup_folder: Text
field_mapping: Text Field 6
show_if:
backup: true
required: true
raw:
preserveOutputOnRerun: true
List of raw options:
Quip will automatically create Repositories or Jobs in external systems. The main purpose is to have GitLab Repository, Jenkins Job, SonarQube Project and WebHook configuration between GitLab and Jenkins.
quip setup
The output of the command will be similar to that one.
======= QUIP (v.1.7.0-BETA) =======
Namespace(action='setup', config=None, debug=False, sonarqube=False, name=None)
INFO:root:Reading template.json file
INFO:root:Project name: ue-AWS Mainframe Modernization
INFO:root:Using config from file : C:\Users\HuseyinGomleksizoglu\.uip_config.yml
==== GITLAB SETUP ====
There is a saved password. To used the saved password just press [Enter]
Please enter Personal Access Key:
Do you want to create Gitlab repository?(Y/n):
INFO:root:Creating GitLab Repository.
Gitlab Groups
=============
(1) cs-uac
(2) integration-prototypes
(3) uac-extensions
(4) uac-extensions/community
(5) uac-extensions/onboarding
(6) uac-extensions/universal-extensions-frameworks
Choose one (1-6) [1]: 1
INFO:root:Checking GitLab if the repository exists.
Repository created cs-uac/ue-aws-mainframe-modernization
Run `git init`
Run `git remote add origin https://gitlab.stonebranch.com/cs-uac/ue-aws-mainframe-modernization.git`
==== JENKINS SETUP ====
INFO:root:Connecting to Jenkins Server.
There is a saved password. To used the saved password just press [Enter]
Please enter Jenkins Password for (h.gomleksizoglu):
INFO:root:Repository doesn't exist and will be created. cs-uac/ue-aws-mainframe-modernization
Do you want to create Jenkins pipeline?(Y/n):
==== SONARQUBE SETUP ====
Do you want to create SonarQube projects?(Y/n):
There is a saved password. To used the saved password just press [Enter]
Please enter Personal Access Key:
INFO:root:URL = https://sonarqube.stonebranch.com/api/projects/create?project=cs-uac-ue-aws-mainframe-modernization&name=cs-uac-ue-aws-mainframe-modernization
SonarQube project created: cs-uac-ue-aws-mainframe-modernization
INFO:root:URL = https://sonarqube.stonebranch.com/api/qualitygates/select?gateName=CS UAC and Community QG&projectKey=cs-uac-ue-aws-mainframe-modernization
SonarQube Quality Gate changed.
The processing logic of quip is like this
originThe processing logic of quip is like this
The processing logic of quip is like this
FAQs
Tool for creating/updating new universal integrations
We found that quip-cli demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.