suma

= Suma: utility for the STEP Unified Model-driven Architecture

== Purpose

SUMA, the "STEP Unified Model-Based Standards Architecture", is a authoring and publication system developed for the management of EXPRESS schemas and its documentation.

This utility is used by the https://github.com/metanorma/iso-10303[ISO 10303 collection] to build the following artifacts:

• the STEP Resource Library (SRL)

== Features

• EXPRESS schema management for STEP standards
• Document collection building and compilation
• EXPRESS links validation and extraction
• Schema listing file generation
• Integration with the Metanorma ecosystem
• Progress tracking for schema loading operations

== Installation

[source,sh]

$ gem install suma

== Usage: CLI

=== General

[source,sh]

Defaults to suma help

$ suma Commands: suma build METANORMA_SITE_MANIFEST # Build collection specified in site manifest (metanorma*.yml) suma reformat EXPRESS_FILE_PATH # Reformat EXPRESS files suma validate SUBCOMMAND ...ARGS # Validate express documents suma generate_schemas METANORMA_MANIFEST_FILE SCHEMA_MANIFEST_FILE # Generate schemas manifest file from Metanorma manifest YAML file suma extract_terms SCHEMA_MANIFEST_FILE GLOSSARIST_OUTPUT_PATH # Extract terms from schema manifest file suma help [COMMAND] # Describe available commands or one specific command

=== Build command

The build command processes a Metanorma site manifest and generates the specified output.

[source,sh]

$ suma build METANORMA_SITE_MANIFEST [options]

Parameters:

METANORMA_SITE_MANIFEST:: This is the path to the Metanorma site manifest, typically metanorma.yml.

Options:

--[no-]compile:: Compile or skip compilation of collection (default: true) --schemas-all-path=PATH, -s PATH:: Generate file that contains all schemas in the collection

The generated schemas-*.yaml file name is derived from the input file name with the word metanorma replaced with schemas.

[example]

.To compile SRL subset test collection [source,sh]

$ bundle exec suma build metanorma-test.yml

.To build SRL collection [source,sh]

$ bundle exec suma build metanorma-srl.yml

.To generate schema listing without compilation [source,sh]

$ bundle exec suma build --no-compile metanorma-srl.yml

=> generates schemas-srl.yml

====

All documents need to have a schemas.yaml in their document root that lists out which schemas the document includes.

=== Reformat command

The reformat command provides utilities for reformatting EXPRESS files.

[source,sh]

$ suma reformat EXPRESS_FILE_PATH [options]

Parameters:

EXPRESS_FILE_PATH:: Path to an EXPRESS file or a folder containing EXPRESS files

Options:

--[no-]recursive, -r:: Select EXPRESS files recursively based on the specified folder path (default: false)

[example]

.To reformat all EXPRESS files under the current directory recursively [source,sh]

$ bundle exec suma reformat pwd -r

====

This command:

• Loads the EXPRESS files specified in the EXPRESS_FILE_PATH
• Reformats and saves the loaded EXPRESS files

=== Validate command

==== General

The validate command groups various validation utilities for EXPRESS documents.

[source,sh]

$ suma validate SUBCOMMAND [options]

Subcommands:

• links - Validate EXPRESS links
• ascii - Check for non-ASCII characters in EXPRESS files

==== Links subcommand

The links subcommand extracts and validates EXPRESS links without creating intermediate files.

[source,sh]

$ suma validate links SCHEMAS_FILE DOCUMENTS_PATH [OUTPUT_FILE]

Parameters:

SCHEMAS_FILE:: Path to the schemas file (default: "schemas-srl.yml")

DOCUMENTS_PATH:: Path to the documents directory (default: "documents")

OUTPUT_FILE:: Path to write validation results (default: "validation_results.txt")

[example]

.To validate EXPRESS links in documents [source,sh]

$ bundle exec suma validate links schemas-srl.yml documents validation_results.txt

====

This command:

• Loads the schemas specified in the SCHEMAS_FILE
• Searches for EXPRESS links in all AsciiDoc files in the DOCUMENTS_PATH
• Validates these links against the loaded schemas
• Writes validation results to the OUTPUT_FILE
• Provides progress bars to track schema loading and link validation

==== ASCII subcommand

The ascii subcommand detects non-ASCII characters in EXPRESS files and reports on those exact lines, providing replacement suggestions.

[source,sh]

$ suma validate ascii EXPRESS_FILE_PATH [options]

Parameters:

EXPRESS_FILE_PATH:: Path to an EXPRESS file or a folder containing EXPRESS files

Options:

--[no-]recursive, -r:: Select EXPRESS files recursively based on the specified folder path (default: false) --[no-]yaml, -y:: Output results in YAML format for machine processing (default: false)

[example]

.To validate all EXPRESS files in a specific directory recursively [source,sh]

$ bundle exec suma validate ascii ../iso-10303/schemas -r

.To validate and output results in YAML format [source,sh]

$ bundle exec suma validate ascii ../iso-10303/schemas -r -y > validation.yml

====

This command:

• Loads the EXPRESS files specified in the EXPRESS_FILE_PATH
• Scans each line for non-ASCII characters
• Reports detailed information about each violation, including: ** Filename and line number ** The exact line content ** Visual indication of the non-ASCII sequence location ** Character details with hexadecimal representation
• Provides specific replacement suggestions: ** For math symbols: provides equivalent AsciiMath notation ** For other non-ASCII characters: provides ISO 10303-11 encoded string literal format
• Displays a summary table showing: ** File path (directory/filename) ** Each non-ASCII symbol found ** Suggested replacement for each symbol ** Number of occurrences of each character ** Totals row showing unique character count and overall occurrences
• Summarizes findings across all scanned files
• Optionally outputs structured data in YAML format with detailed occurrence information

Human-readable output format example:

[source,text]

/path/to/file.exp: Line 42, Column 15: ENTITY some_entity (name: STRING, description: "résumé"); ^^^^^ "é" - Hex: 0xe9, UTF-8 bytes: 0xc3 0xa9 Replacement: ISO 10303-11: "000000E9"

  "s" - Hex: 0x73, UTF-8 bytes: 0x73

  "u" - Hex: 0x75, UTF-8 bytes: 0x75

  "m" - Hex: 0x6d, UTF-8 bytes: 0x6d

  "é" - Hex: 0xe9, UTF-8 bytes: 0xc3 0xa9
  Replacement: ISO 10303-11: "000000E9"

Found 1 non-ASCII sequence(s) in file.exp

Summary: Scanned 3 EXPRESS file(s) Found 1 non-ASCII sequence(s) in 1 file(s)

+------------------+--------------------+-----------------------------+-------------+ | File | Symbol | Replacement | Occurrences | +------------------+--------------------+-----------------------------+-------------+ | path/to/file.exp | "é" (0xe9) | ISO 10303-11: "000000E9" | 2 | +------------------+--------------------+-----------------------------+-------------+ | TOTAL | 1 unique | | 2 | +------------------+--------------------+-----------------------------+-------------+

===== Japanese Character Example

For Japanese characters like 神戸 (Kobe), the command will provide ISO 10303-11 encoded string literal replacements:

[source,text]

"神" - Hex: 0x795e, UTF-8 bytes: 0xe7 0xa5 0x9e Replacement: ISO 10303-11: "0000795E"

"戸" - Hex: 0x6238, UTF-8 bytes: 0xe6 0x88 0xb8 Replacement: ISO 10303-11: "00006238"

===== Math Symbol Example

For mathematical symbols, the command will provide equivalent AsciiMath notation:

[source,text]

"×" - Hex: 0xd7, UTF-8 bytes: 0xc3 0x97 Replacement: AsciiMath: xx

=== Generate schemas command

The suma generate_schemas command generates an EXPRESS schema manifest file containing all schemas of documents referenced in the Metanorma manifest file, recursively.

[source,sh]

$ suma generate_schemas METANORMA_MANIFEST_FILE SCHEMA_MANIFEST_FILE [options]

Parameters:

METANORMA_MANIFEST_FILE:: Path to the Metanorma manifest file (e.g.: "metanorma-smrl-all.yml")

Options:

--exclude_path, -e:: Exclude schemas by pattern (e.g. *_lf.exp)

[example]

.To generate schemas manifest file from Metanorma manifest file [source,sh]

$ bundle exec suma generate_schemas metanorma-smrl-all.yml schemas-smrl-all.yml

=> generates schemas-smrl-all.yml

.To generate schemas manifest file from Metanorma manifest file and exclude schemas with names like *_lf.exp [source,sh]

$ bundle exec suma generate_schemas metanorma-smrl-all.yml schemas-smrl-all.yml -e *_lf.exp

=> generates schemas-smrl-all.yml without schemas with names like *_lf.exp

====

All documents need to have a schemas.yaml in their document root that lists out which schemas the document includes.

=== Extract terms command

The "extract terms" command is implemented for ISO 10303-2, and could also be used for other EXPRESS schema collections that require term extraction for glossary or dictionary applications.

The suma extract_terms command extracts terms from EXPRESS schemas and generates a Glossarist v2 dataset in the output directory. This command processes various types of STEP schemas and creates standardized terminology datasets suitable for glossary and dictionary applications.

[source,sh]

$ suma extract_terms SCHEMA_MANIFEST_FILE GLOSSARIST_OUTPUT_PATH [options]

Parameters:

SCHEMA_MANIFEST_FILE:: Path to the schema manifest file that lists all schemas to process (e.g., "schemas-smrl-all.yml")

GLOSSARIST_OUTPUT_PATH:: Path to the output directory where the Glossarist v2 dataset will be generated

Options:

--language_code, -l:: Language code for the Glossarist dataset (default: "eng")

==== Supported schema types

The command supports extraction from the following EXPRESS schema types:

• ARM (application reference model) - application module schemas ending with _arm
• MIM (module implementation model) - application module schemas ending with _mim
• Resource schemas - General resource schemas
• BOM (business object model) - business object model schemas ending with _bom

NOTE: The "Long Form" EXPRESS schemas (ending with _lf) should be excluded from the manifest file as they do not contain definitions of ENTITY objects.

==== Extracted concepts

The command extracts ENTITY objects from EXPRESS schemas, with the following information:

. a unique identifier in form of {schema_name}.{entity_name} (where entity_name is a slug derived from the ENTITY name); . a generated definition depending on the schema type and entity type; . a "Note to entry" from its first coherent textual element described in Annotated EXPRESS; . source information including: schema name and schema version; . domain in the pattern of: {domain_type}: {schema_name}, where domain_type is one of application module, resource, or business object model.

==== Output format

The command generates a Glossarist v2 compliant dataset with:

• concept/ directory containing concept definition files in YAML
• localized_concept/ directory containing localized concept files in YAML

.To extract terms from a schema manifest file [example]

[source,sh]

$ bundle exec suma extract_terms schemas-smrl-all.yml glossarist_output

=> generates glossarist_output/concept/*.yaml and

glossarist_output/localized_concept/*.yaml

====

.To extract terms from a specific schema subset [example]

[source,sh]

$ bundle exec suma extract_terms schemas-activity-modules.yml terms_output

=> processes only schemas listed in the manifest file

====

== Usage: Ruby

=== General

Suma can be used programmatically in your Ruby applications. The following examples demonstrate common usage patterns.

=== Building collections

[source,ruby]

require 'suma'

Build a collection with default settings

Suma::Processor.run( metanorma_yaml_path: "metanorma-srl.yml", schemas_all_path: "schemas-srl.yml", compile: true, output_directory: "_site" )

Generate schema listing without compilation

Suma::Processor.run( metanorma_yaml_path: "metanorma-srl.yml", schemas_all_path: "schemas-srl.yml", compile: false, output_directory: "_site" )

=== Working with schema configurations

[source,ruby]

require 'suma'

Load schemas using SchemaConfig

schemas_file_path = "schemas-srl.yml" schemas_config = Suma::SchemaConfig::Config.from_yaml(IO.read(schemas_file_path))

Set the initial path to resolve relative paths

schemas_config.set_initial_path(schemas_file_path)

Access schema information

schemas_config.schemas.each do |schema| puts "Schema ID: #{schema.id}" puts "Schema path: #{schema.path}" end

== Copyright and license

Copyright Ribose. BSD 2-clause license.