= LutaML Ruby modeller
https://github.com/lutaml/lutaml-model[image:https://img.shields.io/github/stars/lutaml/lutaml-model.svg?style=social[GitHub Stars]] https://github.com/lutaml/lutaml-model[image:https://img.shields.io/github/forks/lutaml/lutaml-model.svg?style=social[GitHub Forks]] image:https://img.shields.io/github/license/lutaml/lutaml-model.svg[License] image:https://img.shields.io/github/actions/workflow/status/lutaml/lutaml-model/test.yml?branch=main[Build Status] image:https://img.shields.io/gem/v/lutaml-model.svg[RubyGems Version]
== Purpose
Lutaml::Model is the Ruby implementation of the LutaML modeling methodology, for:
It provides simple, flexible and comprehensive mechanisms for defining information models with attributes and types, and the serialization of them to/from serialization formats including JSON, XML, YAML, and TOML.
For serialization formats, it uses an adapter pattern to support multiple libraries for each format, providing flexibility and extensibility for your data modeling needs.
NOTE: The Lutaml::Model modeling Ruby DSL was originally designed to be mostly compatible with the data modeling DSL of https://www.shalerb.org[Shale], a data modeller for Ruby. Lutaml::Model is meant to address advanced needs not currently addressed by Shale. Instructions on how to migrate from Shale to Lutaml::Model are provided in <>.
== Features
toml-rb, tomlib)== Data modeling in a nutshell
Data modeling is the process of creating a data model for the data to be stored in a database or used in an application. It helps in defining the structure, relationships, and constraints of the data, making it easier to manage and use.
Lutaml::Model simplifies data modeling in Ruby by allowing you to define models with attributes and serialize/deserialize them to/from various serialization formats seamlessly.
The Lutaml::Model data modelling approach is as follows:
LutaML Model
│
Has many attributes
│
▼
Attribute
│
Has type of
│
┌──────────┴──────────┐
│ │
Model Value (Leaf)
│ │
Has many attributes Contains one basic value
│ │
┌───────┴─────┐ ┌──────┴──────┐
│ │ │ │
Model Value (Leaf) String Integer
│ Date Boolean
│ Time Float
====
╔═══════════════════════╗ ╔════════════════════════════╗ ║ LutaML Core Model ║ ║ Serialization Models ║ ╚═══════════════════════╝ ╚════════════════════════════╝
╔═══════════════════════╗ ╔══════════════════╗ ╔═══════════════════════╗ ║LutaML Model Class FOO ║ ║LutaML Transformer║ ║LutaML Model Class BAR ║ ╚═══════════════════════╝ ╚══════════════════╝ ╚═══════════════════════╝
Value class, transformation, and serialization formats
[source]╔═════════════════════╗ ╔═════════════════════╗ ╔═════════════════════╗ ║ Studio (Core Model) ║ ║ JSON Model ║ ║ Serialized JSON ║ ╚═════════════════════╝ ╚═════════════════════╝ ╚═════════════════════╝
====
== Installation
Add this line to your application's Gemfile:
And then execute:
Or install it yourself as:
== Model
=== General
There are two ways to define an information model in Lutaml::Model:
Lutaml::Model::Serializable classLutaml::Model::Serialize module[[define-through-inheritance]] === Definition through inheritance
The simplest way to define a model is to create a class that inherits from
Lutaml::Model::Serializable.
The attribute class method is used to define attributes.
require 'lutaml/model'
[[define-through-inclusion]] === Definition through inclusion
If the model class already has a super class that it inherits from, the model
can be extended using the Lutaml::Model::Serialize module.
require 'lutaml/model'
class Kiln < SomeSuperClass include Lutaml::Model::Serialize
=== Comparison
A Serialize / Serializable object can be compared with another object of the
same class using the == operator. This is implemented through the
ComparableModel module.
Two objects are considered equal if they have the same class and all their attributes are equal. This behavior differs from the typical Ruby behavior, where two objects are considered equal only if they have the same object ID.
NOTE: Two Serialize objects will have the same hash value if they have the
same class and all their attributes are equal.
a = Kiln.new(brand: 'Kiln 1', capacity: 100, temperature: 1050) b = Kiln.new(brand: 'Kiln 1', capacity: 100, temperature: 1050) a == b
true
a.hash == b.hash
true
== Value types
=== General types
Lutaml::Model supports the following attribute value types.
Every type has a corresponding Ruby class and a serialization format type.
.Mapping between Lutaml::Model::Type classes, Ruby equivalents and serialization format types |=== | Lutaml::Model::Type | Ruby class | XML | JSON | YAML | Example value
| :string | String | xs:string | string | string | "text"
| :integer | Integer | xs:integer | number | integer | 42
| :float | Float | xs:decimal | number | float | 3.14
| :boolean | TrueClass/FalseClass | xs:boolean | boolean | boolean | true, false
| :date | Date | xs:date | string | string | 2024-01-01 (JSON/YAML "2024-01-01")
| :time_without_date | Time | xs:time | string | string | "12:34:56"
| :date_time | DateTime | xs:dateTime | string | string | "2024-01-01T12:00:00+00:00"
| :time | Time | xs:dateTime | string | string | "2024-01-01T12:00:00+00:00"
| :decimal (optional) | BigDecimal | xs:decimal | number | float | 123.45
| :hash | Hash | complex element | object | map | {key: "value"}
| (nil value) | nil | xs:anyType | null | null | null
// | class | Custom class | complex element | object | map | CustomObject
// | collection: true | Array of type | repeated elements | array | sequence | [obj1, obj2]
// | any
|===
=== Decimal type
WARNING: Decimal is an optional feature.
The Decimal type is a value type that is disabled by default.
NOTE: The reason why the Decimal type is disabled by default is that the
BigDecimal class became optional to the standard Ruby library from Ruby 3.4
onwards. The Decimal type is only enabled when the bigdecimal library is
loaded.
The following code needs to be run before using (and parsing) the Decimal type:
If the bigdecimal library is not loaded, usage of the Decimal type will
raise a Lutaml::Model::TypeNotSupportedError.
=== Custom type
A custom class can be used as an attribute type. The custom class must inherit
from Lutaml::Model::Type::Value or a class that inherits from it.
A class inheriting from the Value class carries the attribute value which
stores the one-and-only "true" value that is independent of serialization
formats.
The minimum requirement for a custom class is to implement the following methods:
self.cast(value):: Assignment of an external value to the Value class to be
set as value. Casts the value to the custom type.
self.serialize(value):: Serializes the custom type to an object (e.g. a
string). Takes the internal value and converts it into an output suitable for
serialization.
class FiveDigitPostCode < Lutaml::Model::Type::String def self.cast(value) value = value.to_s if value.is_a?(Integer)
unless value.is_a?(::String)
raise Lutaml::Model::InvalidValueError, "Invalid value for type 'FiveDigitPostCode'"
end
# Pad zeros to the left
value.rjust(5, '0')
end
def self.serialize(value) value end end
====
=== Serialization of custom types
The serialization of custom types can be made to differ per serialization format
by defining methods in the class definitions. This requires additional methods
than the minimum required for a custom class (i.e. self.cast(value) and
self.serialize(value)).
This is useful in the case when different serialization formats of the same model expect differentiated value representations.
The methods that can be overridden are named:
self.from_{format}(serialized_string):: Deserializes a string of the
serialization format and returns the object to be assigned to the Value class'
value.
to_{format}:: Serializes the object to a string of the serialization format.
The {format} part of the method name is the serialization format in lowercase
(e.g. json, xml, yaml, toml).
Suppose in XML we handle a high-precision date-time type that requires custom serialization methods, but other formats such as JSON do not support this type.
For instance, in the normal DateTime class, the serialized string is
2012-04-07T01:51:37+02:00, and the high-precision format is
2012-04-07T01:51:37.112+02:00.
We create HighPrecisionDateTime class is a custom class that inherits
from Lutaml::Model::Type::DateTime.
class HighPrecisionDateTime < Lutaml::Model::Type::DateTime
self.cast(value) and self.serialize(value) methods2012-04-07T01:51:37.112+02:00def self.from_xml(xml_string) ::DateTime.parse(xml_string) end
def to_xml value.strftime('%Y-%m-%dT%H:%M:%S.%L%:z') end end
An XML snippet with the high-precision date-time type:
When loading the XML snippet, the HighPrecisionDateTime class will be used to
parse the high-precision date-time string.
However, when serializing to JSON, the value will have the high-precision part lost due to the inability of JSON to handle high-precision date-time.
c = Ceramic.from_xml(xml) #<Ceramic:0x0000000104ac7240 @kiln_firing_time=#<HighPrecisionDateTime:0x0000000104ac7240 @value=2012-04-07 01:51:37.112000000 +0200>> c.to_json
{"kilnFiringTime":"2012-04-07T01:51:37+02:00"}
====
== Attributes
=== Basic attributes
An attribute is the basic building block of a model. It is a named value that stores a single piece of data (which may be one or multiple pieces of data).
An attribute only accepts the type of value defined in the attribute definition.
The attribute value type can be one of the following:
Syntax:
Where,
name_of_attribute:: The defined name of the attribute.
Type:: The type of the attribute.
attribute class method to define simple attributes
[example]====
An attribute with a defined value type also accepts values that are of a class that is a subclass of the defined type.
This means that the assigned Type accepts polymorphic classes as long as the
assigned instance is of a class that either inherits from the declared type or
matches it.
class Studio < Lutaml::Model::Serializable attribute :name, :string end
class CeramicStudio < Studio attribute :clay_type, :string end
s = Studio.new(name: 'Pottery Studio') p = PotteryClass.new(studio: s) p.studio
s = CeramicStudio.new(name: 'Ceramic World', clay_type: 'Red') p = PotteryClass.new(studio: s) p.studio
p.studio.name
p.studio.clay_type
====
=== Collection attributes
Define attributes as collections (arrays or hashes) to store multiple values
using the collection option.
collection can be set to:
true:::
The attribute contains an unbounded collection of objects of the declared class.
{min}..{max}:::
The attribute contains a collection of objects of the declared class with a
count within the specified range.
If the number of objects is out of this numbered range,
CollectionCountOutOfRangeError will be raised.
+
[example]0..1, it means that the attribute is optional, it could be empty
or contain one object of the declared class.1.. (equivalent to 1..Infinity), it means that the
attribute must contain at least one object of the declared class and can contain
any number of objects.means that there is a minimum of 5 and a maximum of 10 objects of the declared class. If the count of values for the attribute is less then 5 or greater then 10, theCollectionCountOutOfRangeError` will be raised.Syntax:
collection option to define a collection attribute
[example]Studio.new
address count is
0, must be between 1 and 2 (Lutaml::Model::CollectionCountOutOfRangeError)Studio.new({ address: ["address 1", "address 2", "address 3"] })
address count is
3, must be between 1 and 2 (Lutaml::Model::CollectionCountOutOfRangeError)Studio.new({ address: ["address 1"] }).potters
[]
Studio.new({ address: ["address 1"] }).address
["address 1"]
Studio.new(address: ["address 1"], potters: ['John Doe', 'Jane Doe']).potters
['John Doe', 'Jane Doe']
====
=== Derived attributes
A derived attribute is computed dynamically based on an instance method instead of storing a static value. It is defined using the method: option.
Syntax:
class Invoice < Lutaml::Model::Serializable attribute :subtotal, :float attribute :tax, :float attribute :total, method: :total_value
def total_value subtotal + tax end end
i = Invoice.new(subtotal: 100.0, tax: 12.0) i.total #=> 112.0
====
=== Choice attributes
The choice directive allows specifying that elements from the specified range are included.
NOTE: Attribute-level definitions are supported. This can be used with both
key_value and xml mappings.
Syntax:
Where,
min:: The minimum number of elements that must be included.
max:: The maximum number of elements that can be included.
block:: The block of elements that must be included. The block can contain
multiple attribute and choice directives.
choice directive to define a set of attributes with a range
[example]class Studio < Lutaml::Model::Serializable choice(min: 1, max: 3) do choice(min: 1, max: 2) do attribute :prefix, :string attribute :forename, :string end
attribute :completeName, :string
This means that the Studio class must have at least one and at most three
attributes.
completeName.prefix and forename attributes or just the forename attribute.completeName is optional.
======= Importable models for reuse
An importable model is a model that can be imported into another model using the
import_* directive.
Such a model is specified by setting the model's XML serialization configuration
with the no_root directive.
As a result, the model can be imported into another model using the following directives:
import_model:: imports both attributes and mappings.
import_model_attributes:: imports only attributes.
import_model_mappings:: imports only mappings.
NOTE: This feature only works with XML for now. The import order determines how elements and attributes are overwritten.
Models with no_root can only be parsed through parent models.
Direct calling NoRootModel.from_xml will raise a NoRootMappingError.
Namespaces are not supported in importable models. If namespace is defined with
no_root, NoRootNamespaceError will raise.
class GroupOfItems < Lutaml::Model::Serializable attribute :name, :string attribute :type, :string attribute :code, :string
xml do no_root sequence do map_element "name", to: :name map_element "type", to: :type, namespace: "http://www.example.com", prefix: "ex1" end map_attribute "code", to: :code end end
class ComplexType < Lutaml::Model::Serializable attribute :tag, AttributeValueType attribute :content, :string attribute :group, :string import_model_attributes GroupOfItems
xml do root "GroupOfItems" map_attribute "tag", to: :tag map_content to: :content map_element :group, to: :group import_model_mappings GroupOfItems end end
class SimpleType < Lutaml::Model::Serializable import_model GroupOfItems end
parsed = GroupOfItems.from_xml(xml)
Lutaml::Model::NoRootMappingError: "GroupOfItems has
no_root, it allowed only for reusable models"
====
=== Value validation
==== General
There are several mechanisms to validate attribute values in Lutaml::Model.
[[attribute-enumeration]] ==== Values of an enumeration
An attribute can be defined as an enumeration by using the values directive.
The values directive is used to define acceptable values in an attribute. If
any other value is given, a Lutaml::Model::InvalidValueError will be raised.
Syntax:
The values set inside the values: option can be of any type, but they must
match the type of the attribute. The values are compared using the == operator,
so the type must implement the == method.
Also, If all the elements in values directive are strings then lutaml-model add some enum convenience methods, for each of the value the following three methods are added
value1: will return value if setvalue1?: will return true if value is set, false otherwisevalue1=: will set the value of name_of_attribute equal to value1 if truthy value is given, and remove it otherwise.values directive to define acceptable values for an attribute (basic types)
[example]GlazeTechnique.new(name: "Celadon").name
"Celadon"
GlazeTechnique.new(name: "Raku").name
"Raku"
GlazeTechnique.new(name: "Majolica").name
"Majolica"
GlazeTechnique.new(name: "Earthenware").name
Lutaml::Model::InvalidValueError: Invalid value for attribute 'name'
====
The values can be Serialize objects, which are compared using the ==
and the hash methods through the Lutaml::Model::ComparableModel module.
values directive to define acceptable values for an attribute (Serializable objects)
[example]class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :firing_temperature, :integer end
CeramicCollection.new(featured_piece: Ceramic.new(type: "Porcelain", firing_temperature: 1300)).featured_piece
Ceramic:0x0000000104ac7240 @type="Porcelain", @firing_temperature=1300
CeramicCollection.new(featured_piece: Ceramic.new(type: "Bone China", firing_temperature: 1300)).featured_piece
Lutaml::Model::InvalidValueError: Invalid value for attribute 'featured_piece'
====
Serialize provides a validate method that checks if all its attributes have
valid values. This is necessary for the case when a value is valid at the
component level, but not accepted at the aggregation level.
If a change has been made at the component level (a nested attribute has
changed), the aggregation level needs to call the validate method to verify
acceptance of the newly updated component.
validate method to check if all attributes have valid values
[example]collection = CeramicCollection.new(featured_piece: Ceramic.new(type: "Porcelain", firing_temperature: 1300)) collection.featured_piece.firing_temperature = 1400
No error raised in changed nested attribute
collection.validate
Lutaml::Model::InvalidValueError: Invalid value for attribute 'featured_piece'
====
==== String values restricted to patterns
An attribute that accepts a string value accepts value validation using regular expressions.
Syntax:
pattern option to restrict the value of an attribute
[example]In this example, the color attribute takes hex color values such as #ccddee.
A regular expression can be used to validate values assigned to the attribute.
In this case, it is /^#([A-Fa-f0-9]{6}|[A-Fa-f0-9]{3})$/.
Glaze.new(color: '#ff0000').color
"#ff0000"
Glaze.new(color: '#ff000').color
Lutaml::Model::InvalidValueError: Invalid value for attribute 'color'
====
=== Attribute value defaults
Specify default values for attributes using the default option.
The default option can be set to a value or a lambda that returns a value.
Syntax:
default option to set a default value for an attribute
[example]Glaze.new.color
"Clear"
Glaze.new.temperature
1050
====
The "default behavior" (pun intended) is to not render a default value if the current value is the same as the default value.
=== Attribute as raw string
An attribute can be set to read the value as raw string for XML, by using the raw: true option.
Syntax:
raw option to read raw value for an XML attribute
[example]For the following XML snippet:
Person.from_xml(xml)
<Person:0x0000000107a3ca70
@description="\n A <b>fictional person</b> commonly used as a <i>placeholder name</i>.\n ",
@element_order=["text", "name", "text", "description", "text"],
@name="John Doe",
@ordered=nil>
====
== Serialization model mappings
=== General
Lutaml::Model allows you to translate a data model into serialization models of various serialization formats including XML, JSON, YAML, and TOML.
Depending on the serialization format, different methods are supported for defining serialization and deserialization mappings.
Serialization model mappings are defined under the xml, json, yaml, and
toml blocks.
xml, json, yaml, toml and key_value blocks to define serialization mappings
[source,ruby]class Example < Lutaml::Model::Serializable xml do # ... end
json do # ... end
yaml do # ... end
toml do # ... end
=== XML
==== Setting root element name
The root method sets the root element tag name of the XML document.
If root is not given, then the snake-cased class name will be used as the
root.
[example]
Sets the tag name for <example> in XML <example>...</example>.
Syntax:
example
[example]Example.new.to_xml #
====
==== Ommiting root element
The root element can be omitted by using the no_root method.
When no_root is used, only map_element can be used because without a root
element there cannot be attributes.
Syntax:
class NameAndCode < Lutaml::Model::Serializable attribute :name, :string attribute :code, :string
ID-001parsed = NameAndCode.from_xml(xml)
<NameAndCode:0x0000000107a3ca70 @code="ID-001", @name="Name">
parsed.to_xml
ID-001Name
====
[[xml-map-all]] ==== Mapping all XML content
The map_all tag in XML mapping captures and maps all content within an XML
element into a single attribute in the target Ruby object.
The use case for map_all is to tell Lutaml::Model to not parse the content of
the XML element at all, and instead handle it as an XML string.
NOTE: The corresponding method for key-value formats is at <>.
WARNING: Notice that usage of mapping all will lead to incompatibility between serialization formats, i.e. the raw string content will not be portable as objects are across different formats.
This is useful in the case where the content of an XML element is not to be handled by a Lutaml::Model::Serializable object.
This feature is commonly used with custom methods or a custom model object to handle the content.
This includes:
The map_all tag is exclusive and cannot be combined with other mappings
(map_element, map_content) except for map_attribute for the same element,
ensuring it captures the entire inner XML content.
NOTE: An error is raised if map_all is defined alongside any other mapping in
the same XML mapping context.
Syntax:
map_all
[example]class ExampleMapping < Lutaml::Model::Serializable attribute :description, :string
parsed = ExampleMapping.from_xml(xml) puts parsed.all_content
====
==== Mapping elements
The map_element method maps an XML element to a data model attribute.
[example]
To handle the <name> tag in <example><name>John Doe</name></example>.
The value will be set to John Doe.
Syntax:
name tag to the name attribute
[example]class Example < Lutaml::Model::Serializable attribute :name, :string
Example.from_xml(xml) #<Example:0x0000000104ac7240 @name="John Doe"> Example.new(name: "John Doe").to_xml #John Doe
====
If an element is mapped to a model object with the XML root tag name set, the
mapped tag name will be used as the root name, overriding the root name.
class RecordDate < Lutaml::Model::Serializable attribute :content, :string
xml do root "recordDate" map_content to: :content end end
class OriginInfo < Lutaml::Model::Serializable attribute :date_issued, RecordDate, collection: true
RecordDate.new(date: "2021-01-01").to_xml #2021-01-01 OriginInfo.new(date_issued: [RecordDate.new(date: "2021-01-01")]).to_xml #2021-01-01
====
==== Mapping attributes
The map_attribute method maps an XML attribute to a data model attribute.
Syntax:
map_attribute to map the value attribute
[example]The following class will parse the XML snippet below:
class Example < Lutaml::Model::Serializable attribute :value, :integer
Example.from_xml(xml) #<Example:0x0000000104ac7240 @value=12> Example.new(value: 12).to_xml #
====
The map_attribute method does not inherit the root element's namespace.
To specify a namespace for an attribute, please explicitly declare the
namespace and prefix in the map_attribute method.
The following class will parse the XML snippet below:
class Attribute < Lutaml::Model::Serializable attribute :value, :integer
Attribute.from_xml(xml) #<Attribute:0x0000000109436db8 @value=20> Attribute.new(value: 20).to_xml #<example xmlns:xl="http://www.tech.co/XMI\" xl:value="20"/>
====
==== Mapping content
Content represents the text inside an XML element, inclusive of whitespace.
The map_content method maps an XML element's content to a data model
attribute.
Syntax:
map_content to map content of the description tag
[example]The following class will parse the XML snippet below:
class Example < Lutaml::Model::Serializable attribute :description, :string
Example.from_xml(xml) #<Example:0x0000000104ac7240 @description="John Doe is my moniker."> Example.new(description: "John Doe is my moniker.").to_xml #John Doe is my moniker.
====
==== CDATA nodes
CDATA is an XML feature that allows the inclusion of text that may contain characters that are unescaped in XML.
While CDATA is not preferred in XML, it is sometimes necessary to handle CDATA nodes for both input and output.
NOTE: The W3C XML Recommendation explicitly encourages escaping characters over usage of CDATA.
Lutaml::Model supports the handling of CDATA nodes in XML in the following behavior:
. When an attribute contains a CDATA node with no text: ** On reading: The node (CDATA or text) is read as its value. ** On writing: The value is written as its native type.
. When an XML mapping sets cdata: true on map_element or map_content:
** On reading: The node (CDATA or text) is read as its value.
** On writing: The value is written as a CDATA node.
. When an XML mapping sets cdata: false on map_element or map_content:
** On reading: The node (CDATA or text) is read as its value.
** On writing: The value is written as a text node (string).
Syntax:
cdata to map CDATA content
[example]The following class will parse the XML snippet below:
class Example < Lutaml::Model::Serializable attribute :name, :string attribute :description, :string attribute :title, :string attribute :note, :string
Example.from_xml(xml) #<Example:0x0000000104ac7240 @name="John" @description="here is the description" @title="Lutaml" @note="Careful"> Example.new(name: "John", description: "here is the description", title: "Lutaml", note: "Careful").to_xml #LutamlCareful
====
==== Example for mapping
The following class will parse the XML snippet below:
class Ceramic < Lutaml::Model::Serializable attribute :name, :string attribute :description, :string attribute :temperature, :integer
Ceramic.from_xml(xml) #<Ceramic:0x0000000104ac7240 @name="Porcelain Vase", @description=" with celadon glaze.", @temperature=1200> Ceramic.new(name: "Porcelain Vase", description: " with celadon glaze.", temperature: 1200).to_xml #Porcelain Vase with celadon glaze.
====
==== Namespaces
[[root-namespace]] ===== Namespace at root
The namespace method in the xml block sets the namespace for the root
element.
Syntax:
namespace method to set the namespace for the root element
[example]class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, :string
Ceramic.from_xml(xml_file) #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear"> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml #PorcelainClear
====
namespace method to set a prefixed namespace for the root element
[example]class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, :string
Ceramic.from_xml(xml_file) #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear"> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml #<cer:Ceramic xmlns="http://example.com/ceramic">cer:TypePorcelain</cer:Type>cer:GlazeClear</cer:Glaze></cer:Ceramic>
====
===== Namespace on attribute
If the namespace is defined on a model attribute that already has a namespace, the mapped namespace will be given priority over the one defined in the class.
Syntax:
namespace:: The XML namespace used by this element
prefix:: The XML namespace prefix used by this element (optional)
namespace option to set the namespace for an element
[example]In this example, glz will be used for Glaze if it is added inside the
Ceramic class, and glaze will be used otherwise.
class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, Glaze
xml do root 'Ceramic' namespace 'http://example.com/ceramic'
map_element 'Type', to: :type
map_element 'Glaze', to: :glaze, namespace: 'http://example.com/glaze', prefix: "glz"
end end
class Glaze < Lutaml::Model::Serializable attribute :color, :string attribute :temperature, :integer
xml do root 'Glaze' namespace 'http://example.com/old_glaze', 'glaze'
map_element 'color', to: :color
map_element 'temperature', to: :temperature
Using the original Glaze class namespace
Glaze.new(color: "Clear", temperature: 1050).to_xml #<glaze:Glaze xmlns="http://example.com/old_glaze">Clear1050</glaze:Glaze>
Using the Ceramic class namespace for Glaze
Ceramic.from_xml(xml_file) #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=1050>> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear", temperature: 1050)).to_xml #Porcelain<glz:Glaze xmlns="http://example.com/glaze">Clear1050</glz:Glaze>
====
[[namespace-inherit]]
===== Namespace with inherit option
The inherit option is used at the element level to inherit the namespace from
the root element.
Syntax:
inherit option to inherit the namespace from the root element
[example]In this example, the Type element will inherit the namespace from the root.
class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, :string attribute :color, :string
Ceramic.from_xml(xml_file) #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear", @color="navy-blue"> Ceramic.new(type: "Porcelain", glaze: "Clear", color: "navy-blue").to_xml #<cera:Ceramic xmlns:cera="http://example.com/ceramic"
====
[[mixed-content]] ==== Mixed content
In XML there can be tags that contain content mixed with other tags and where whitespace is significant, such as to represent rich text.
My name is John Doe, and I'm 28 years old
====
To map this to Lutaml::Model we can use the mixed option in either way:
NOTE: This feature is not supported by Shale.
To specify mixed content, the mixed: true option needs to be set at the
xml block's root method.
Syntax:
mixed to treat root as mixed content
[example]class Paragraph < Lutaml::Model::Serializable attribute :bold, :string, collection: true # allows multiple bold tags attribute :italic, :string
xml do root 'p', mixed: true
map_element 'bold', to: :bold
map_element 'i', to: :italic
Paragraph.from_xml("
My name is John Doe, and I'm 28 years old
") #<Paragraph:0x0000000104ac7240 @bold="John Doe", @italic="28"> Paragraph.new(bold: "John Doe", italic: "28").to_xml #My name is John Doe, and I'm 28 years old
====
// TODO: How to create mixed content from #new?
[[ordered-content]] ==== Ordered content
ordered: true maintains the order of XML Elements, while mixed: true preserves the order of XML Elements and Content.
NOTE: When both options are used, mixed: true takes precedence.
To specify ordered content, the ordered: true option needs to be set at the
xml block's root method.
Syntax:
ordered to treat root as ordered content
[example]class RootOrderedContent < Lutaml::Model::Serializable attribute :bold, :string attribute :italic, :string attribute :underline, :string
instance = RootOrderedContent.from_xml(xml) #<RootOrderedContent:0x0000000104ac7240 @bold="bell", @italic="384,400 km", @underline="Moon"> instance.to_xml #Moon384,400 kmbell
Without Ordered True:
class RootOrderedContent < Lutaml::Model::Serializable attribute :bold, :string attribute :italic, :string attribute :underline, :string
instance = RootOrderedContent.from_xml(xml) #<RootOrderedContent:0x0000000104ac7240 @bold="bell", @italic="384,400 km", @underline="Moon"> instance.to_xml #\n bell\n 384,400 km\n Moon\n
====
==== Sequence
The sequence directive specifies that the defined attributes must appear in a
specified order in XML.
NOTE: Sequence only supports map_element mappings.
Syntax:
The appearance of the elements in the XML document must match the order defined
in the sequence block. In this case, the <xml_element_name_1> element
should appear before the <xml_element_name_2> element.
sequence keyword to define a set of elements in desired order.
[example]class Kiln < Lutaml::Model::Serializable attribute :id, :string attribute :name, :string attribute :type, :string attribute :color, :string
xml do sequence do map_element :id, to: :id map_element :name, to: :name map_element :type, to: :type map_element :color, to: :color end end end
class KilnCollection < Lutaml::Model::Serializable attribute :kiln, Kiln, collection: 1..2
parsed = Kiln.from_xml(xml)
#<Kiln:0x0000000104ac7240 @id="1", @name="Nick", @type="Hard", @color="Black">, #<Kiln:0x0000000104ac7240 @id="2", @name="John", @type="Soft", @color="White"> #]
Nick 1 Black Hard HERE > parsed = Kiln.from_xml(bad_xml) # => Lutaml::Model::ValidationError: Element 'name' is out of order in 'kiln' element ---- ====bad_xml = <<~HERE
[[xml-schema-location]]
==== Automatic support of xsi:schemaLocation
The
https://www.w3.org/TR/xmlschema-1/#xsi_schemaLocation[W3C "XMLSchema-instance"]
namespace describes a number of attributes that can be used to control the
behavior of XML processors. One of these attributes is xsi:schemaLocation.
The xsi:schemaLocation attribute locates schemas for elements and attributes
that are in a specified namespace. Its value consists of pairs of a namespace
URI followed by a relative or absolute URL where the schema for that namespace
can be found.
Usage of xsi:schemaLocation in an XML element depends on the declaration of
the XML namespace of xsi, i.e.
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". Without this namespace
LutaML will not be able to serialize the xsi:schemaLocation attribute.
NOTE: It is most commonly attached to the root element but can appear further down the tree.
The following snippet shows how xsi:schemaLocation is used in an XML document:
LutaML::Model supports the xsi:schemaLocation attribute in all XML
serializations by default, through the schema_location attribute on the model
instance object.
xsi:schemaLocation attribute in XML serialization
[example]In this example, the xsi:schemaLocation attribute will be automatically
supplied without the explicit need to define in the model, and allows for
round-trip serialization.
class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, :string attribute :color, :string
xml do root 'Ceramic' namespace 'http://example.com/ceramic', 'cera' map_element 'Type', to: :type, namespace: :inherit map_element 'Glaze', to: :glaze map_attribute 'color', to: :color, namespace: 'http://example.com/color', prefix: 'clr' end end
c = Ceramic.from_xml(xml_content) => #<Ceramic:0x00000001222bdd60 ... schema_loc = c.schema_location #<Lutaml::Model::SchemaLocation:0x0000000122773760 ... schema_loc => #<Lutaml::Model::SchemaLocation:0x0000000122773760 @namespace="http://www.w3.org/2001/XMLSchema-instance", @original_schema_location="http://example.com/ceramic http://example.com/ceramic.xsd http://example.com/color http://example.com/color.xsd", @prefix="xsi", @schema_location= [#<Lutaml::Model::Location:0x00000001222bd018 @location="http://example.com/ceramic.xsd", @namespace="http://example.com/ceramic">, #<Lutaml::Model::Location:0x00000001222bcfc8 @location="http://example.com/color.xsd", @namespace="http://example.com/color">]> new_c = Ceramic.new(type: "Porcelain", glaze: "Clear", color: "navy-blue", schema_location: schema_loc).to_xml puts new_c
====
NOTE: For details on xsi:schemaLocation, please refer to the
https://www.w3.org/TR/xmlschema-1/#xsi_schemaLocation[W3C XML standard].
==== Character encoding
===== General
Lutaml::Model XML adapters use a default encoding of UTF-8 for both input and
output.
Serialization data to be parsed (deserialization) and serialization data to be exported (serialization) may be in a different character encoding than the default encoding used by the Lutaml::Model XML adapter. This mismatch may lead to incorrect data reading or incompatibilities when exporting data.
The possible values for setting character encoding to are:
A valid encoding value, e.g. UTF-8, Shift_JIS, ASCII;
nil to use the default encoding of the adapter. The behavior differs based
on the adapter used.
** Nokogiri: UTF-8. The encoding is set to the default encoding of the Nokogiri library,
which is UTF-8.
** Oga: UTF-8. The encoding is set to the default encoding of the Oga library, which
uses UTF-8.
** Ox: ASCII-8bit. The encoding is set to the default encoding of the Ox library, which uses
ASCII-8bit.
When the encoding option is not set, the default encoding of UTF-8 is
used.
===== Serialization character encoding (exporting)
====== General
There are two ways to set the character encoding of the XML document during serialization:
Instance setting::
Setting the instance-level encoding option by setting
ModelClassInstance.encoding('...'). This setting only affects serialization.
Per-export setting::
Setting the encoding option when calling for serialization action using the
ModelClassInstance.to_xml(..., encoding: ...) method.
[[encoding-instance-setting]] ====== Instance setting
The encoding value of an instance sets the character encoding of the XML
document during serialization.
Syntax:
Where,
ModelClassInstance:: An instance of the class that inherits from
Lutaml::Model::Serializable.
{encoding_value}:: The encoding of the output data.
class JapaneseCeramic < Lutaml::Model::Serializable attribute :glaze_type, :string attribute :description, :string
instance = JapaneseCeramic.new(glaze_type: "志野釉", description: "東京国立博物館コレクションの篠茶碗「橋本」（桃山時代）") #=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」（桃山時代）">
instance.encoding = "Shift_JIS" #=> "Shift_JIS"
serialization_output = instance.to_xml #=> #\x{5FD8}\x{91CE}\x{91C9}\x{6771}\x{4EAC}\x{56FD}\x{7ACB}\x{535A}\x{7269}\x{9928}\x{30B3}\x{30EC}\x{30AF}\x{30B7}\x{30E7}\x{30F3}\x{306E}\x{7BC0}\x{8336}\x{7897}\x{300C}\x{6A4B}\x{672C}\x{300D}\x{FF08}\x{6853}\x{5C71}\x{6642}\x{4EE3}\x{FF09}
serialization_output.encoding #=> "Shift_JIS"
====
====== Per-export setting
The encoding option is used in the ModelClass#to_xml(..., encoding: ...)
call to set the character encoding of the XML document during serialization.
The per-export encoding setting supersedes the instance-level encoding setting.
Syntax:
Where,
ModelClassInstance:: An instance of the class that inherits from
Lutaml::Model::Serializable.
{encoding_value}:: The encoding of the output data.
The following class will parse the XML snippet below:
class Ceramic < Lutaml::Model::Serializable attribute :potter, :string attribute :description, :string attribute :temperature, :integer
ceramic_instance = Ceramic.new(potter: "John & Jane", description: " A ∑ series of ∏ porcelain µ vases.") #<Ceramic:0x0000000104ac7240 @potter="John & Jane", @description=" A ∑ series of ∏ porcelain µ vases.">
ceramic_parsed = Ceramic.from_xml(xml) #<Ceramic:0x0000000104ac7242 @potter="John & Jane", @description=" A ∑ series of ∏ porcelain µ vases.">
ceramic_parsed == ceramic_instance
true
ceramic_instance.to_xml #John & Jane A ∑ series of ∏ porcelain µ vases.
ceramic_instance.to_xml(encoding: nil) #John & Jane A ∑ series of ∏ porcelain µ vases.
ceramic_instance.to_xml(encoding: "ASCII") #John & Jane A ∑ series of ∏ porcelain µ vases.
====
to_xml overrides instance encoding
[example]class JapaneseCeramic < Lutaml::Model::Serializable attribute :glaze_type, :string attribute :description, :string
instance = JapaneseCeramic.new(glaze_type: "志野釉", description: "東京国立博物館コレクションの篠茶碗「橋本」（桃山時代）") #=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」（桃山時代）">
instance.encoding = "Shift_JIS" #=> "Shift_JIS"
serialization_output = instance.to_xml(encoding: "UTF-8") #=> #志野釉東京国立博物館コレクションの篠茶碗「橋本」（桃山時代）
serialization_output.encoding #=> "UTF-8"
====
===== Deserialization character encoding (parsing)
The character encoding of the XML document being parsed is specified using the
encoding option when the ModelClass.from_{format}(...) is called.
Syntax:
Where,
ModelClass:: The class that inherits from Lutaml::Model::Serializable.
{format}:: The format of the input data, e.g. xml, json, yaml, toml.
string_in_format:: The input data in the specified format.
{encoding_value}:: The encoding of the input data.
encoding option during parsing data not encoded in the default encoding (UTF-8)
[example]Using the definition of JapaneseCeramic at <>.
This XML snippet is in Shift-JIS.
instance = JapaneseCeramic.from_xml(xml, encoding: "Shift_JIS") #=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」（桃山時代）">
instance.encoding #=> "Shift_JIS"
serialization_output = instance.to_xml(encoding: "UTF-8") #=> #志野釉東京国立博物館コレクションの篠茶碗「橋本」（桃山時代） serialization_output.encoding #=> "UTF-8"
====
encoding option is not set, the default encoding of the adapter is used
[example]Using the definition of JapaneseCeramic at <>.
This XML snippet is in UTF-8.
In adapters that use a default encoding of UTF-8, the content is parsed
properly.
instance = JapaneseCeramic.from_xml(xml, encoding: nil) #=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」（桃山時代）"> instance.encoding #=> "UTF-8" serialization_output = instance.to_xml #=> #志野釉東京国立博物館コレクションの篠茶碗「橋本」（桃山時代） serialization_output.encoding #=> "UTF-8"
In adapters that use a default encoding of ASCII-8bit, the content becomes
malformed.
instance = JapaneseCeramic.from_xml(xml, encoding: nil) #=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="�菑�", @description="�東京国立博物館コレクションの篠茶碗�橋本�桃山時代�"> instance.encoding #=> "ASCII-8bit" serialization_output = instance.to_xml #=> #�菑��東京国立博物館コレクションの篠茶碗�橋本�桃山時代� serialization_output.encoding #=> "ASCII-8bit"
====
Using the definition of JapaneseCeramic at <>.
This XML snippet is in UTF-8.
JapaneseCeramic.from_xml(xml, encoding: "Shift_JIS") #=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="�菑���p���P", @description="�東京国立博物館コレクションの篠茶碗�橋本�桃山時代�">
====
=== Key value data models
==== General
Key-value data models like JSON, YAML, and TOML all share a similar structure where data is stored as key-value pairs.
Lutaml::Model works with these formats in a similar way.
==== Mapping
The map method is used to define key-value mappings.
Syntax:
==== Unified mapping
The key_value method is a streamlined way to map all attributes for
serialization into key-value formats including JSON, YAML, and TOML.
If there is no definite differentiation between the key value formats, the
key_value method simplifies defining mappings and improves code readability.
map method to define the same mappings across all key-value formats
[example]This example shows how to define a key-value data model with the key_value
method which maps the same attributes across all key-value formats.
class CeramicModel < Lutaml::Model::Serializable attribute :color, :string attribute :glaze, :string attribute :description, :string
key_value do map :color, to: color map :glz, to: :glaze map :desc, to: :description end
CeramicModel.from_json(json) #<CeramicModel:0x0000000104ac7240 @color="Navy Blue", @glaze="Clear", @description="A ceramic with a navy blue color and clear glaze."> CeramicModel.new(color: "Navy Blue", glaze: "Clear", description: "A ceramic with a navy blue color and clear glaze.").to_json #{"color"=>"Navy Blue", "glz"=>"Clear", "desc"=>"A ceramic with a navy blue color and clear glaze."}
====
==== Specific format mappings
Specific key value formats can be mapping independently of other formats, including:
json for the JSON formatyaml for the YAML formattoml for the TOML formatmap method to define key-value mappings per format
[example]class Example < Lutaml::Model::Serializable attribute :name, :string attribute :value, :integer
json do map 'name', to: :name map 'value', to: :value end
yaml do map 'name', to: :name map 'value', to: :value end
Example.from_json(json) #<Example:0x0000000104ac7240 @name="John Doe", @value=28> Example.new(name: "John Doe", value: 28).to_json #{"name"=>"John Doe", "value"=>28}
====
[[key-value-map-all]] ==== Mapping all key-value content
The map_all tag captures and maps all content within a serialization format
into a single attribute in the target Ruby object.
The use case for map_all is to tell Lutaml::Model to not parse the content at
all, and instead handle it as a raw string.
NOTE: The corresponding method for XML is at <>.
WARNING: Notice that usage of mapping all will lead to incompatibility between serialization formats, i.e. the raw string content will not be portable as objects are across different formats.
This is useful when the content needs to be handled as-is without parsing into individual attributes.
The map_all tag is exclusive and cannot be combined with other mappings,
ensuring it captures the entire content.
NOTE: An error is raised if map_all is defined alongside any other mapping in
the same mapping context.
Syntax:
map_all to capture all content across different formats
[example]class Document < Lutaml::Model::Serializable attribute :content, :string
json do map_all to: :content end
yaml do map_all to: :content end
sections:
The content is preserved exactly as provided:
doc = Document.from_json(json_content) puts doc.content
"{"sections":[{"title":"Introduction","text":"Chapter 1"},{"title":"Conclusion","text":"Final chapter"}],"metadata":{"author":"John Doe","date":"2024-01-15"}}"
doc = Document.from_yaml(yaml_content) puts doc.content
"sections:\n - title: Introduction\n text: Chapter 1\n - title: Conclusion\n text: Final chapter\nmetadata:\n author: John Doe\n date: 2024-01-15\n"
====
==== Nested attribute mappings
The map method can also be used to map nested key-value data models
by referring to a Lutaml::Model class as an attribute class.
class Glaze < Lutaml::Model::Serializable attribute :color, :string attribute :temperature, :integer
json do map 'color', to: :color map 'temperature', to: :temperature end end
class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, Glaze
Ceramic.from_json(json) #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=1050>> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear", temperature: 1050)).to_json #{"type"=>"Porcelain", "glaze"=>{"color"=>"Clear", "temperature"=>1050}}
====
==== Collection with keyed elements (keyed collection)
===== General
NOTE: This feature is for key-value data model serialization and deserialization only.
The map method with the root_mappings option is used for key-value data that
is keyed using an attribute value.
In other words, the key of a key-value pair in a collection is actually the value of an attribute that belongs to the value.
Simply put, the following two data structures are considered to have the same data:
id attribute
[source,yaml]id attribute value located inside each element
[source,yaml]There are key difference between these two data structures:
The <<collection-keyed-by-value,keyed object>> (first data structure) ensures
uniqueness of the id attribute value across the collection, while the
<<collection-unkeyed-by-value,array>> (second data structure) does not.
The value of the id attribute in the first data structure exists outside
of the formal structure of the data object, instead, it only exists at the
collection level. On the other hand, the value exists inside the structure of
the data object in the second data structure.
The map method with the root_mappings option, in practice, parses the first
data structure in the same way that you would access / manipulate the second
data structure, while retaining the serialization semantics of using an
attribute as key.
As a result, usage of lutaml-model across both types of collections are identical (except when serialized).
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable attribute :name_of_attribute, AttributeValueType, collection: true
json | yaml | toml | key_value do
map to: :name_of_attribute, <1>
root_mappings: { <2>
# :key is a reserved keyword
value_type_attribute_name_for_key: :key, <3>
# :value is a reserved keyword (and optional)
value_type_attribute_name_for_value: :value, <4>
# [path name] represents the path to access the value in the
# serialization data model to be assigned to
# AttributeValueType.value_type_attribute_name_for_custom_type
value_type_attribute_name_for_custom_type: [path name] <5>
}
end
end
<1> The map option indicates that this class represents the root of the
serialization object being passed in. The name_of_attribute is the name
of the attribute that will hold the collection data. (Mandatory)
<2> The root_mappings keyword specifies what the collection key represents and
and value for model. (Mandatory)
<3> The key keyword specifies the attribute name of the individual collection
object type that represents its key used in the collection. (Mandatory)
<4> The value keyword specifies the attribute name of the individual collection
object type that represents its data used in the collection. (Optional, if
not specified, the entire object is used as the value.)
<5> The value_type_attribute_name_for_custom_type is the name of the attribute
inside the individual collection object (AttributeValueType) that will hold
the value accessible in the serialization data model fetched at [path name].
The mapping syntax here is similar to that of <> except
that the :key and :value keywords are allowed in addition to {path}.
There are 3 cases when working with a keyed collection:
. Case 1: Only move the "key" into the collection object.
. Case 2: Move the "key" into the collection object, override all other
mappings. Maps :key and another attribute, then we override all the other
mappings (clean slate)
. Case 3: Move the "key" into the collection object to an attribute, map the entire "value" to another attribute of the collection object.
===== Case 1: Only move the "key" into the collection object
In this case, the "key" of the keyed collection is moved into the collection object, and all other mappings are left as they are.
When the "key" is moved into the collection object, the following happens:
The root_mappings option should only contain one mapping, and the mapping
must lead to the :key keyword.
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable attribute :name_of_attribute, AttributeValueType, collection: true
json | yaml | toml | key_value do map to: :name_of_attribute, root_mappings: { value_type_attribute_name_for_key: :key, <1> } end end
<1> The :key keyword specifies that the "key" of the keyed collection maps
to the value_type_attribute_name_for_key attribute of the collection's
instance object (i.e. AttributeValueType).
map with root_mappings (only key) to map a keyed collection into individual models
[example]Given this data:
A model can be defined for this YAML as follows:
class Ceramic < Lutaml::Model::Serializable attribute :ceramic_id, :string attribute :ceramic_name, :string
key_value do map 'id', to: :ceramic_id map 'name', to: :ceramic_name end end
class CeramicCollection < Lutaml::Model::Serializable attribute :ceramics, Ceramic, collection: true
ceramics attribute
root_mappings: {
# The key of an object in this collection is mapped to the ceramic_id
# attribute of the Ceramic object.
ceramic_id: :key # "key" is a reserved keyword
}
end
endceramic_collection = CeramicCollection.from_yaml(yaml) #<CeramicCollection:0x0000000104ac7240 @ceramics= [#<Ceramic:0x0000000104ac6e30 @ceramic_id="vase1", @ceramic_name="Imperial Vase">, #<Ceramic:0x0000000104ac58f0 @ceramic_id="bowl2", @ceramic_name="18th Century Bowl">]
id attribute is:key, everyting is mapped to the Ceramic object using the mappings defined in the Ceramic class.first_ceramic = ceramic_collection.ceramics.first puts first_ceramic.to_yaml =>
ceramic_id attribute is used to key the data,puts ceramic_collection.to_yaml =>
ceramic_id attribute is used tomap with root_mappings method innew_collection = CeramicCollection.new(ceramics: [ Ceramic.new(ceramic_id: "vase1", ceramic_name: "Imperial Vase"), Ceramic.new(ceramic_id: "bowl2", ceramic_name: "18th Century Bowl") ]) puts new_collection.to_yaml =>
====
===== Case 2: Mapping the key and complex values
In this use case, the "key" of the keyed collection is moved into the collection object, and all other mappings are overridden.
When more than one mapping rule exists in the root_mappings option, the
root_mappings option will override all other mappings in the collection object.
When the "key" is moved into the collection object, the following happens:
The "key" of the keyed collection maps to a particular attribute of the collection's instance object.
The data of the "value" of the keyed collection have their own mappings
overridden by the new mapping rules of the root_mappings option.
The root_mappings option can contain more than one mapping, with one of
the mapping rules leading to the :key keyword.
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable attribute :name_of_attribute, AttributeValueType, collection: true
json | yaml | toml | key_value do map to: :name_of_attribute, root_mappings: { value_type_attribute_name_for_key: :key, <1> value_type_attribute_name_for_value_data_1: "serialization_format_name_1", <2> value_type_attribute_name_for_value_data_2: "serialization_format_name_2", value_type_attribute_name_for_value_data_3: ["path name", ...] <3> # ... } end end
class AttributeValueType < Lutaml::Model::Serializable attribute :value_type_attribute_name_for_key, :string attribute :value_type_attribute_name_for_value_data_1, :string attribute :value_type_attribute_name_for_value_data_2, SomeType attribute :value_type_attribute_name_for_value_data_3, MoreType
<1> The :key keyword specifies that the "key" of the keyed collection maps
to the value_type_attribute_name_for_key attribute of the collection's
instance object (i.e. AttributeValueType).
<2> The serialization_format_name_1 target specifies that the
serialization_format_name_2 key of the keyed collection value maps to the
value_type_attribute_name_for_value_data_1 attribute of the collection's
instance object.
<3> The [path name] target specifies to fetch from [path name] in the
serialization data model to be assigned to the
value_type_attribute_name_for_value_data_3 attribute of the collection's
instance object.
When the root_mappings mapping contains more than one mapping rule that is not
to :key or :value, the root_mappings mapping will override all other
mappings in the collection object. This means that unmapped attributes in
root_mappings will not be incorporated in the collection instance objects.
map with root_mappings (key and complex value) to map a keyed collection into individual models
[example]A model can be defined for this YAML as follows:
class CeramicDetails < Lutaml::Model::Serializable attribute :name, :string attribute :insignia, :string
key_value do map 'name', to: :name map 'insignia', to: :insignia end end
class Ceramic < Lutaml::Model::Serializable attribute :ceramic_id, :string attribute :ceramic_type, :string attribute :ceramic_details, CeramicDetails attribute :ceramic_urn, :string
key_value do map 'id', to: :ceramic_id map 'type', to: :ceramic_type map 'details', to: :ceramic_details map 'urn', to: :ceramic_urn end end
class CeramicCollection < Lutaml::Model::Serializable attribute :ceramics, Ceramic, collection: true
ceramics attribute
root_mappings: {
# The key of an object in this collection is mapped to the ceramic_id
# attribute of the Ceramic object.
# (e.g. vase1, bowl2)
ceramic_id: :key,
ceramic_type: :type,
ceramic_details: "details",
ceramic_urn: ["urn", "primary"]
}
end
endThe output becomes:
ceramics_collection = CeramicCollection.from_yaml(yaml) => #<CeramicCollection:0x0000000107a2cf30 @ceramics= [#<Ceramic:0x0000000107a2cf30 @ceramic_id="vase1", @ceramic_type="vase", @ceramic_details= #<CeramicDetails:0x0000000107a2cf30 @name="Imperial Vase", @insignia="Tang Tianbao">, @ceramic_urn="urn:ceramic:vase:vase1">, #<Ceramic:0x0000000107a2cf30 @ceramic_id="bowl2", @ceramic_type="bowl", @ceramic_details= #<CeramicDetails:0x0000000107a2cf30 @name="18th Century Bowl", @insignia="Ming Wanli"> @ceramic_urn="urn:ceramic:bowl:bowl2">]
first_ceramic = ceramics_collection.ceramics.first puts first_ceramic.to_yaml =>
new_collection = CeramicCollection.new(ceramics: [ Ceramic.new(ceramic_id: "vase1", ceramic_type: "vase", ceramic_urn: "urn:ceramic:vase:vase1", ceramic_details: CeramicDetails.new( name: "Imperial Vase", insignia: "Tang Tianbao") ), Ceramic.new(ceramic_id: "bowl2", ceramic_type: "bowl", ceramic_urn: "urn:ceramic:vase:bowl2", ceramic_details: CeramicDetails.new( name: "18th Century Bowl", insignia: "Ming Wanli") ) ]) new_collection.to_yaml
====
===== Case 3: Mapping the key and delegating value to an inner object
In this use case, the "key" of the keyed collection is moved into the collection object to an attribute, and the entire "value" of the keyed collection is mapped to another attribute of the collection object.
When the "key" is moved into the collection object, the following happens:
The "key" of the keyed collection maps to a particular attribute of the collection's instance object.
The data of the "value" of the keyed collection will be entirely mapped into an attribute of the collection's instance object.
The original mapping of the "value" attribute of the collection's instance object is retained.
The root_mappings option should only contain two mappings, and the mappings
must lead to both the :key and :value keywords.
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable attribute :name_of_attribute, AttributeValueType, collection: true
json | yaml | toml | key_value do map to: :name_of_attribute, root_mappings: { value_type_attribute_name_for_key: :key, <1> value_type_attribute_name_for_value: :value <2> } end end
<1> The :key keyword specifies that the "key" of the keyed collection maps
to the value_type_attribute_name_for_key attribute of the collection's
instance object (i.e. AttributeValueType).
<2> The :value keyword specifies that the entire "value" of the keyed
collection maps to the value_type_attribute_name_for_value attribute of the
collection's instance object (i.e. SomeObject).
When the root_mappings mapping contains more than one mapping rule, the
root_mappings mapping will override all other mappings in the collection
object. This means that unmapped attributes in root_mappings will not be
incorporated in the collection instance objects.
map with root_mappings (key and value) to map a keyed collection into individual models
[example]Given this data:
A model can be defined for this YAML as follows:
class CeramicDetails < Lutaml::Model::Serializable attribute :name, :string attribute :insignia, :string
key_value do map 'name', to: :name map 'insignia', to: :insignia end end
class Ceramic < Lutaml::Model::Serializable attribute :ceramic_id, :string attribute :ceramic_details, CeramicDetails
key_value do map 'id', to: :ceramic_id map 'details', to: :ceramic_details end end
class CeramicCollection < Lutaml::Model::Serializable attribute :ceramics, Ceramic, collection: true
ceramics attribute
root_mappings: {
# The key of an object in this collection is mapped to the ceramic_id
# attribute of the Ceramic object.
# (e.g. vase1, bowl2)
ceramic_id: :key,
# The value of an object in this collection is mapped to the
# ceramic_details attribute of the Ceramic object.
# (e.g. name: 18th Century Bowl, insignia: "Ming Wanli"
ceramic_details: :value
}
end
endceramic_collection = CeramicCollection.from_yaml(yaml) #<CeramicCollection:0x0000000104ac7240 @ceramics= [#<Ceramic:0x0000000104ac6e30 @ceramic_id="vase1", @ceramic_details= #<CeramicDetails:0x0000000104ac6e30 @name="Imperial Vase", @insignia="Tang Tianbao">, #<Ceramic:0x0000000104ac58f0 @ceramic_id="bowl2", @ceramic_details= #<CeramicDetails:0x0000000104ac58f0 @name="18th Century Bowl", @insignia="Ming Wanli">]
id attribute isfirst_ceramic = ceramic_collection.ceramics.first puts first_ceramic.to_yaml =>
ceramic_id attribute is used to key the data,puts ceramic_collection.to_yaml =>
ceramic_id attribute is used tomap with root_mappings method innew_collection = CeramicCollection.new(ceramics: [ Ceramic.new(ceramic_id: "vase1", ceramic_details: CeramicDetails.new( name: "Imperial Vase", insignia: "Tang Tianbao") ), Ceramic.new(ceramic_id: "bowl2", ceramic_details: CeramicDetails.new( name: "18th Century Bowl", insignia: "Ming Wanli") ) ]) puts new_collection.to_yaml =>
====
[[attribute-extraction]] ==== Attribute extraction
NOTE: This feature is for key-value data model serialization only.
The child_mappings option is used to extract results from a key-value
serialization data model (JSON, YAML, TOML) into a Lutaml::Model::Serializable
object (collection or not).
The values are extracted from the key-value data model using the list of keys provided.
Syntax:
class SomeObject < Lutaml::Model::Serializable attribute :name_of_attribute, AttributeValueType, collection: true
<1> The value_type_attribute_name_1 is the attribute name in the
AttributeValueType model. The value of this attribute will be assigned the key
of the hash in the key-value data model.
<2> The path_to_value_1 is an array of keys that represent the path to the
value in the key-value serialization data model. The keys are used to extract the value from
the key-value serialization data model and assign it to the attribute in the
AttributeValueType model.
+
The path_to_value is in a nested array format with each value a symbol or a
string, where each symbol represents a key to traverse down. The last key in the
path is the value to be extracted.
The following JSON contains 2 keys in schema named engine and gearbox.
engine schema is [:components, :engine] and for
the gearbox schema is [:components, :gearbox].In path_to_value, the :key and :value are reserved instructions used to
assign the key or value of the serialization data respectively as the value to
the attribute.
In the following JSON content, the path_to_value for the object keys named
engine and gearbox will utilize the :key keyword to assign the key of the
object as the value of a designated attribute.
====
If a specified value path is not found, the corresponding attribute in the model
will be assigned a nil value.
nil when the path_to_value is not found
[example]In the following JSON content, the path_to_value of [:extras, :sunroof] and
[:extras, :drinks_cooler] at the object "gearbox" would be set to nil.
====
child_mappings option to extract values from a key-value data model
[example]The following JSON contains 2 keys in schema named foo and bar.
<1> The keys foo and bar are to be mapped to the id attribute.
<2> The nested path.link and path.name keys are used as the link and
name attributes, respectively.
A model can be defined for this JSON as follows:
class Schema < Lutaml::Model::Serializable attribute :id, :string attribute :link, :string attribute :name, :string end
class ChildMappingClass < Lutaml::Model::Serializable attribute :schemas, Schema, collection: true
The output becomes:
ChildMappingClass.from_json(json) #<ChildMappingClass:0x0000000104ac7240 @schemas= [#<Schema:0x0000000104ac6e30 @id="foo", @link="link one", @name="one">, #<Schema:0x0000000104ac58f0 @id="bar", @link="link two", @name="two">]> ChildMappingClass.new(schemas: [Schema.new(id: "foo", link: "link one", name: "one"), Schema.new(id: "bar", link: "link two", name: "two")]).to_json #{"schemas"=>{"foo"=>{"path"=>{"link"=>"link one", "name"=>"one"}}, {"bar"=>{"path"=>{"link"=>"link two", "name"=>"two"}}}}}
In this example:
The key of each schema (foo and bar) is mapped to the id attribute.
The nested path.link and path.name keys are mapped to the link and
name attributes, respectively.
====
[[separate-serialization-model]] === Separate serialization model
The Serialize module can be used to define only serialization mappings for a
separately defined model (a Ruby class).
Syntax:
class Foo < Lutaml::Model::Serializable model {DataModelClass}
model method to define serialization mappings for a separate modelclass Ceramic attr_accessor :type, :glaze
def name "#{type} with #{glaze}" end end
class CeramicSerialization < Lutaml::Model::Serializable model Ceramic
Ceramic.new(type: "Porcelain", glaze: "Clear").name
"Porcelain with Clear"
CeramicSerialization.from_xml(xml) #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear"> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml #PorcelainClear
====
=== Rendering empty attributes and collections
By default, empty attributes and collections are not rendered in the output.
To render empty attributes and collections, use the render_nil option.
Syntax:
render_nil option to render empty attributes
[example]class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, :string
xml do map_element 'type', to: :type, render_nil: true map_element 'glaze', to: :glaze end
Ceramic.new.to_json
{ 'type': null }
Ceramic.new(type: "Porcelain", glaze: "Clear").to_json
{ 'type': 'Porcelain', 'glaze': 'Clear' }
Ceramic.new.to_xml
Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml
PorcelainClear
====
render_nil option to render empty attribute collections
[example]class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glazes, :string, collection: true
xml do map_element 'type', to: :type, render_nil: true map_element 'glazes', to: :glazes, render_nil: true end
Ceramic.new.to_json
{ 'type': null, 'glazes': [] }
Ceramic.new(type: "Porcelain", glazes: ["Clear"]).to_json
{ 'type': 'Porcelain', 'glazes': ['Clear'] }
Ceramic.new.to_xml
Ceramic.new(type: "Porcelain", glazes: ["Clear"]).to_xml
PorcelainClear
====
=== Rendering default values (forced rendering of default values)
By default, attributes with default values are not rendered if the current value is the same as the default value.
In certain cases, it is necessary to render the default value even if the
current value is the same as the default value. This is achieved by setting the
render_default option to true.
Syntax:
attribute :name_of_attribute, Type, default: -> { value }
xml do map_element 'name_of_attribute', to: :name_of_attribute, render_default: true map_attribute 'name_of_attribute', to: :name_of_attribute, render_default: true end
render_default option to force encoding the default value
[example]class Glaze < Lutaml::Model::Serializable attribute :color, :string, default: -> { 'Clear' } attribute :opacity, :string, default: -> { 'Opaque' } attribute :temperature, :integer, default: -> { 1050 } attribute :firing_time, :integer, default: -> { 60 }
xml do root "glaze" map_element 'color', to: :color map_element 'opacity', to: :opacity, render_default: true map_attribute 'temperature', to: :temperature map_attribute 'firingTime', to: :firing_time, render_default: true end
====
render_default: true are rendered when the value is identical to the default
[example]glaze_new = Glaze.new puts glaze_new.to_xml
puts glaze_new.to_json
====
render_default: true with non-default values are rendered
[example]glaze = Glaze.new(color: 'Celadon', opacity: 'Semitransparent', temperature: 1300, firing_time: 90) puts glaze.to_xml
puts glaze.to_json
====
=== Advanced attribute mapping
==== Multiple mappings to single attribute
The mapping methods support multiple names mapping to a single attribute using an array of names.
Syntax:
json | yaml | toml | key_value do map ["name1", "name2"], to: :attribute_name end
When serializing, the first element in the array of mapped names is always used as the output name.
class CustomModel < Lutaml::Model::Serializable attribute :full_name, Lutaml::Model::Type::String attribute :color, Lutaml::Model::Type::String attribute :id, Lutaml::Model::Type::String
json do map ["name", "custom_name"], with: { to: :name_to_json, from: :name_from_json } map ["color", "shade"], with: { to: :color_to_json, from: :color_from_json } end
xml do root "CustomModel" map_element ["name", "custom-name"], with: { to: :name_to_xml, from: :name_from_xml } map_element ["color", "shade"], with: { to: :color_to_xml, from: :color_from_xml } map_attribute ["id", "identifier"], to: :id end
def name_to_json(model, doc) doc["name"] = "JSON Model: #{model.full_name}" end
def name_from_json(model, value) model.full_name = value&.sub(/^JSON Model: /, "") end
def color_to_json(model, doc) doc["color"] = model.color.upcase end
def color_from_json(model, value) model.color = value&.downcase end
def name_to_xml(model, parent, doc) el = doc.create_element("name") doc.add_text(el, "XML Model: #{model.full_name}") doc.add_element(parent, el) end
def name_from_xml(model, value) model.full_name = value.sub(/^XML Model: /, "") end
def color_to_xml(model, parent, doc) el = doc.create_element("color") doc.add_text(el, model.color.upcase) doc.add_element(parent, el) end
model = CustomModel.from_json(json) model.full_name
"Vase"
model.color
"blue"
====
==== Attribute mapping delegation
Delegate attribute mappings to nested objects using the delegate option.
Syntax:
delegate option to map attributes to nested objects
[example]The following class will parse the JSON snippet below:
class Glaze < Lutaml::Model::Serializable attribute :color, :string attribute :temperature, :integer
json do map 'color', to: :color map 'temperature', to: :temperature end end
class Ceramic < Lutaml::Model::Serializable attribute :type, :string attribute :glaze, Glaze
Ceramic.from_json(json) #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=nil>> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear")).to_json #{"type"=>"Porcelain", "color"=>"Clear"}
====
NOTE: The corresponding keyword used by Shale is receiver: instead of
delegate:.
=== Value Transformations
The transform option allows defining import/export transformations at both attribute and mapping levels:
class Person < Lutaml::Model::Serializable
attribute :name, :string, transform: { export: ->(value) { value.upcase }, import: ->(value) { value.downcase } }
json do map "fullName", to: :name, transform: { export: ->(value) { "Dr. #{value}" }, import: ->(value) { value.gsub("Dr. ", "") } } end
The transformation precedence is:
This allows flexible value transformations without needing format-specific custom methods:
person = Person.new(name: "john")
person.to_json # => {"fullName": "Dr. john"}
Person.from_json({ "fullName" => "Dr. john"}.to_json).name # => john
The transform option supports:
export: Transform value during serializationimport: Transform value during deserialization==== Attribute serialization with custom methods
===== General
Define custom methods for specific attribute mappings using the with: key for
each serialization mapping block for from and to.
===== XML serialization with custom methods
Syntax:
with: key to define custom serialization methods for XML
[example]The following class will parse the XML snippet below:
class Metadata < Lutaml::Model::Serializable attribute :category, :string attribute :identifier, :string end
class CustomCeramic < Lutaml::Model::Serializable attribute :name, :string attribute :size, :integer attribute :description, :string attribute :metadata, Metadata
xml do map_element "Name", to: :name, with: { to: :name_to_xml, from: :name_from_xml } map_attribute "Size", to: :size, with: { to: :size_to_xml, from: :size_from_xml } map_content with: { to: :description_to_xml, from: :description_from_xml } map_element :metadata, to: :metadata, with: { to: :metadata_to_xml, from: :metadata_from_xml } end
def name_to_xml(model, parent, doc) el = doc.create_element("Name") doc.add_text(el, "XML Masterpiece: #{model.name}") doc.add_element(parent, el) end
def name_from_xml(model, value) model.name = value.sub(/^XML Masterpiece: /, "") end
def size_to_xml(model, parent, doc) doc.add_attribute(parent, "Size", model.size + 3) end
def size_from_xml(model, value) model.size = value.to_i - 3 end
def description_to_xml(model, parent, doc) doc.add_text(parent, "XML Description: #{model.description}") end
def description_from_xml(model, value) model.description = value.join.strip.sub(/^XML Description: /, "") end
def metadata_to_xml(model, parent, doc) metadata_el = doc.create_element("metadata") category_el = doc.create_element("category") identifier_el = doc.create_element("identifier")
doc.add_text(category_el, model.metadata.category)
doc.add_text(identifier_el, model.metadata.identifier)
doc.add_element(metadata_el, category_el)
doc.add_element(metadata_el, identifier_el)
doc.add_element(parent, metadata_el)
end
def metadata_from_xml(model, value) model.metadata ||= Metadata.new
model.metadata.category = value["elements"]["category"].text
model.metadata.identifier = value["elements"]["identifier"].text
CustomCeramic.from_xml(xml) #<CustomCeramic:0x0000000108d0e1f8 @element_order=["text", "Name", "text", "Size", "text"], @name="Masterpiece: Vase", @ordered=nil, @size=12, @description="A beautiful ceramic vase", @metadata=#<Metadata:0x0000000105ad52e0 @category="Metadata", @identifier="123">> puts CustomCeramic.new(name: "Vase", size: 12, description: "A beautiful vase", metadata: Metadata.new(category: "Glaze", identifier: 15)).to_xml
def custom_method_from_xml(model, value) instance = value.node # Lutaml::Model::XmlAdapter::AdapterElement
instance = value.node.adapter_node # Adapter::Element
When building a model from XML in custom methods, if the value parameter is a mapping_hash, then it allows access to the parsed XML structure through value.node which can be converted to an XML string using to_xml.
NOTE: For NokogiriAdapter, we can also call to_xml on value.node.adapter_node.
value
{"text"=>["\n ", "\n ", "\n "], "elements"=>{"category"=>{"text"=>"Metadata"}}}
value.to_xml
undefined_method
to_xml
value.node
#<Lutaml::Model::XmlAdapter::NokogiriElement:0x0000000107656ed8
#<Lutaml::Model::XmlAdapter::OxElement:0x0000000107584f78
value.node.to_xml #Metadata
====
==== Separate Serialization Model With Custom Methods
The following class will parse the XML snippet below:
class CustomModelChild attr_accessor :street, :city end
class CustomModelChildMapper < Lutaml::Model::Serializable model CustomModelChild
attribute :street, Lutaml::Model::Type::String attribute :city, Lutaml::Model::Type::String
xml do map_element :street, to: :street map_element :city, to: :city end end
class CustomModelParentMapper < Lutaml::Model::Serializable attribute :first_name, Lutaml::Model::Type::String attribute :child_mapper, CustomModelChildMapper
xml do map_element :first_name, to: :first_name map_element :CustomModelChild, with: { to: :child_to_xml, from: :child_from_xml } end
def child_to_xml(model, parent, doc) child_el = doc.create_element("CustomModelChild") street_el = doc.create_element("street") city_el = doc.create_element("city")
doc.add_text(street_el, model.child_mapper.street)
doc.add_text(city_el, model.child_mapper.city)
doc.add_element(child_el, street_el)
doc.add_element(child_el, city_el)
doc.add_element(parent, child_el)
end
def child_from_xml(model, value) model.child_mapper ||= CustomModelChild.new
model.child_mapper.street = value["elements"]["street"].text
model.child_mapper.city = value["elements"]["city"].text
instance = CustomModelParentMapper.from_xml(xml) #<CustomModelParent:0x0000000107c9ca68 @child_mapper=#<CustomModelChild:0x0000000107c95218 @city="London", @street="Oxford Street">, @first_name="John"> CustomModelParentMapper.to_xml(instance) #<first_name>John</first_name>Oxford StreetLondon
to_xml is called on the mapper class, not on model instance.===== Key-value data model serialization with custom methods
with: key to define custom serialization methods
[example]The following class will parse the JSON snippet below:
class CustomCeramic < Lutaml::Model::Serializable attribute :name, :string attribute :size, :integer
json do map 'name', to: :name, with: { to: :name_to_json, from: :name_from_json } map 'size', to: :size end
def name_to_json(model, doc) doc["name"] = "Masterpiece: #{model.name}" end
CustomCeramic.from_json(json) #<CustomCeramic:0x0000000104ac7240 @name="Vase", @size=12> CustomCeramic.new(name: "Vase", size: 12).to_json #{"name"=>"Masterpiece: Vase", "size"=>12}
====
== Importing data models
=== General
Lutaml::Model provides a way to import data models defined from various formats into the LutaML data modeling system.
Data model languages supported are:
The following figure illustrates the process of importing an XML Schema model to create LutaML core models. Once the LutaML core models are created, they can be used to parse and generate XML documents according to the imported XML Schema model.
Today, the LutaML core models are written into Ruby files, which can be used to parse and generate XML documents according to the imported XML Schema. This is to be changed so that the LutaML core models are directly loaded and interpreted.
╔════════════════════════════╗ ╔═══════════════════════╗ ║ Serialization Models ║ ║ Core Model ║ ╚════════════════════════════╝ ╚═══════════════════════╝
[[xml-schema-to-model-files]] === XML Schema (XSD)
W3C XSD is a schema language designed to define the structure of XML documents, alongside other XML schema languages like DTD, RELAX NG, and Schematron.
Lutaml::Model supports the import of XSD schema files to define information models that can be used to parse and generate XML documents.
Specifically, the Lutaml::Model::Schema#from_xml method loads XML Schema files
(XSD, *.xsd) and generates Ruby files (*.rb) that inherit from
Lutaml::Model::Serializable that are saved to disk.
Syntax:
<1> The xsd_schema is the XML Schema string to be converted to model files.
<2> The options hash is an optional argument.
options:: Optional hash containing potentially the following key-values.
output_dir::: The directory where the model files will be saved. If not
provided, a default directory named lutaml_models_<timestamp> is created.
+
[example]
"path/to/directory"
create_files::: A boolean argument (false by default) to create files directly in the specified directory as defined by the output_dir option.
+
[example]
create_files: (true | false)
load_classes::: A boolean argument (false by default) to load generated classes before returning them.
+
[example]
load_classes: (true | false)
namespace::: The namespace of the schema. This will be added in the
Lutaml::Model::Serializable file's xml do block.
+
[example]
http://example.com/namespace
prefix::: The prefix of the namespace provided in the namespace option.
+
[example]
example-prefix
location::: The URL or path of the directory containing all the files of the
schema. For more information, refer to the
link:https://www.w3.org/TR/xmlschema-1/#include[XML Schema specification].
+
[example]
"http://example.com/example.xsd"
+
[example]
"path/to/schema/directory"
NOTE: If both create_files and load_classes are provided, the create_files argument will take priority and generate files without loading them!
The generated LutaML models consists of two different kind of Ruby classes depending on the XSD schema:
XSD "SimpleTypes":: converted into classes that inherit from
Lutaml::Model::Type::Value, which define the data types with restrictions and
other validations of these values.
XSD "ComplexTypes":: converted into classes that inherit from
Lutaml::Model::Serializable that model according to the defined structure.
Lutaml::Model uses the https://github.com/lutaml/lutaml-xsd[`lutaml-xsd` gem] to
automatically resolve the include and import elements, enabling
Lutaml-Model to generate the corresponding model files.
This auto-resolving feature allows seamless integration of these files into your models without the need for manual resolution of includes and imports.
Lutaml::Model::Schema#from_xml to convert an XML Schema to model filesxsd_schema = <<~XSD <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"> /* your schema here */ </xs:schema> XSD options = {
output_dir: 'path/to/directory', namespace: 'http://example.com/namespace', prefix: "example-prefix", location: "http://example.com/example.xsd",
create_files: true, # Default: false
load_classes: true, # Default: false }
====
You could also directly load the generated Ruby files into your application by requiring them.
====
== Validation
=== General
Lutaml::Model provides a way to validate data models using the validate and
validate! methods.
The validate method sets an errors array in the model instance that
contains all the validation errors. This method is used for checking the
validity of the model silently.
The validate! method raises a Lutaml::Model::ValidationError that contains
all the validation errors. This method is used for forceful validation of the
model through raising an error.
Lutaml::Model supports the following validation methods:
collection:: Validates collection size range.values:: Validates the value of an attribute from a set of fixed values.choice :: Validates that attribute specified within defined rangeThe following class will validate the degree_settings attribute to ensure that
it has at least one element and that the description attribute is one of the
values in the set [one, two, three].
class Klin < Lutaml::Model::Serializable attribute :name, :string attribute :degree_settings, :integer, collection: (1..) attribute :description, :string, values: %w[one two three] attribute :id, :integer attribute :age, :integer
choice(min: 1, max: 1) do choice(min: 1, max: 2) do attribute :prefix, :string attribute :forename, :string end
attribute :nick_name, :string
end
xml do map_element 'name', to: :name map_attribute 'degree_settings', to: :degree_settings end end
klin = Klin.new(name: "Klin", degree_settings: [100, 200, 300], description: "one", prefix: "Ben") klin.validate
klin = Klin.new(name: "Klin", degree_settings: [], description: "four", prefix: "Ben", nick_name: "Smith") klin.validate
e = klin.validate!
e.errors
====
=== Custom validation
To add custom validation, override the validate method in the model class.
Additional errors should be added to the errors array.
The following class validates the degree_settings attribute when the type is
glass to ensure that the value is less than 1300.
class Klin < Lutaml::Model::Serializable attribute :name, :string attribute :type, :string, values: %w[glass ceramic] attribute :degree_settings, :integer, collection: (1..)
def validate errors = super if type == "glass" && degree_settings.any? { |d| d > 1300 } errors << Lutaml::Model::Error.new("Degree settings for glass must be less than 1300") end end end
klin = Klin.new(name: "Klin", type: "glass", degree_settings: [100, 200, 1400]) klin.validate
====
== Liquid template access
WARNING: The Liquid template feature is optional. To enable it, please
explicitly require the liquid gem.
The https://shopify.github.io/liquid/[Liquid template language] is an open-source template language developed by Shopify and written in Ruby.
Lutaml::Model::Serializable objects can be safely accessed within Liquid
templates through a to_liquid method that converts the objects into
Liquid::Drop instances.
Liquid::Drop objects so
inner attributes can be accessed using the Liquid dot notation.NOTE: Every Lutaml::Model::Serializable class extends the Liquefiable module
which generates a corresponding Liquid::Drop class.
NOTE: Methods defined in the Lutaml::Model::Serializable class are not
accessible in the Liquid template.
.Using to_liquid to convert model instances into corresponding Liquid drop instances
class Ceramic < Lutaml::Model::Serializable attribute :name, :string attribute :temperature, :integer end
ceramic = Ceramic.new({ name: "Porcelain Vase", temperature: 1200 }) ceramic_drop = ceramic.to_liquid
puts ceramic_drop.name
puts ceramic_drop.temperature
====
class Ceramic < Lutaml::Model::Serializable attribute :name, :string attribute :temperature, :integer end
sample.yml:
ceramics:
template.liquid:
{% for ceramic in ceramic_collection.ceramics %}
ceramic_collection = CeramicCollection.from_yaml(File.read("sample.yml"))
template = Liquid::Template.parse(File.read("template.liquid"))
output = template.render("ceramic_collection" => ceramic_collection) puts output
====
class Glaze < Lutaml::Model::Serializable attribute :color, :string attribute :opacity, :string end
class CeramicWork < Lutaml::Model::Serializable attribute :name, :string attribute :glaze, Glaze end
class CeramicCollection < Lutaml::Model::Serializable attribute :ceramics, Ceramic, collection: true end
ceramic_work = CeramicWork.new({ name: "Celadon Bowl", glaze: Glaze.new({ color: "Jade Green", opacity: "Translucent" }) }) ceramic_work_drop = ceramic_work.to_liquid
puts ceramic_work_drop.name
puts ceramic_work_drop.glaze.color
puts ceramic_work_drop.glaze.opacity
ceramics.yml:
ceramics:
templates/_ceramics.liquid:
NOTE: render is a Liquid tag that renders a partial template, by default
Liquid uses the pattern _%s.liquid to find the partial template. Here
ceramic refers to the file at templates/_ceramic.liquid.
templates/_ceramic.liquid:
require 'liquid'
template = Liquid::Template.new
file_system = Liquid::LocalFileSystem.new('templates/') template.registers[:file_system] = file_system
file_system.pattern to see what it loads)template.parse(file_system.read_template_file('ceramics'))
ceramic_collection = CeramicCollection.from_yaml(File.read("ceramics.yml"))
output = template.render("ceramic_collection" => ceramic_collection) puts output
====
== Adapters
=== General
Lutaml::Model uses an adapter pattern to support multiple libraries for each serialization format.
Lutaml::Model supports the following serialization formats:
You will need to specify the configuration for the adapter you want to use. The easiest way is to copy and paste the following configuration into your code.
The configuration is as follows:
require 'lutaml/model' require 'lutaml/model/xml_adapter/nokogiri_adapter' require 'lutaml/model/json_adapter/standard_json_adapter' require 'lutaml/model/toml_adapter/toml_rb_adapter' require 'lutaml/model/yaml_adapter/standard_yaml_adapter'
You can also provide the adapter type by using symbols like
require 'lutaml/model'
NOTE: By default yaml_adapter_type and json_adapter_type are set to
:standard_yaml and :standard_json respectively.
=== XML
Lutaml::Model supports the following XML adapters:
Nokogiri::
(default)
Popular libxml based XML parser for Ruby.
Requires native extensions (i.e. compiled C code).
Requires the nokogiri gem.
Oga::
(optional)
Pure Ruby XML parser.
Does not require native extensions and is suitable for
https://opalrb.com[Opal] (Ruby on JavaScript).
Requires the oga gem.
Ox::
(optional)
Fast XML parser and object serializer for Ruby, implemented partially in C.
Requires native extensions (i.e. compiled C code).
Requires the ox gem.
require 'lutaml/model'
require 'lutaml/model'
require 'lutaml/model'
=== YAML
Lutaml::Model supports only one YAML adapter.
YAML:: (default) The Psych YAML parser and emitter for Ruby. Included in the Ruby standard library.
require 'lutaml/model'
=== JSON
Lutaml::Model supports the following JSON adapters:
JSON:: (default) The standard JSON library for Ruby. Included in the Ruby standard library.
MultiJson::
(optional)
A gem that provides a common interface to multiple JSON libraries.
Requires the multi_json gem.
require 'lutaml/model'
require 'lutaml/model'
=== TOML
Lutaml::Model supports the following TOML adapters:
Toml-rb::
(default)
A TOML parser and serializer for Ruby that is compatible with the TOML v1.0.0
specification.
Requires the toml-rb gem.
Tomlib::
(optional)
Toml-rb fork that is compatible with the TOML v1.0.0 specification, but with
additional features.
Requires the tomlib gem.
require 'lutaml/model'
require 'lutaml/model'
== Comparison with Shale
Lutaml::Model is a serialization library that is similar to Shale, but with some differences in implementation.
[cols="a,a,a,a",options="header"] |=== | Feature | Lutaml::Model | Shale | Notes
| Data model definition | 3 types:
<<define-through-inheritance,Inherit from Lutaml::Model::Serializable>>
<<define-through-inclusion,Include Lutaml::Model::Serialize>>
<<separate-serialization-model,Separate serialization model class>> | 2 types:
Inherit from Shale::Mapper
Custom model class |
| Value types
| Lutaml::Model::Type includes: Integer, String, Float, Boolean, Date, DateTime, Time, Decimal, Hash.
| Shale::Type includes: Integer, String, Float, Boolean, Date, Time.
| Lutaml::Model supports additional value types Decimal, DateTime and Hash.
| Configuration
| Lutaml::Model::Config
| Shale.{type}_adapter
| Lutaml::Model uses a configuration block to set the serialization adapters.
| Custom serialization methods
| :with, on individual attributes
| :using, on entire object/document
| Lutaml::Model uses the :with keyword for custom serialization methods.
| Serialization formats | XML, YAML, JSON, TOML | XML, YAML, JSON, TOML, CSV | Lutaml::Model does not support CSV.
| Validation | Supports collection range, fixed values, and custom validation | Requires implementation |
| Adapter support | XML (Nokogiri, Ox, Oga), YAML, JSON (JSON, MultiJson), TOML (Toml-rb, Tomlib) | XML (Nokogiri, Ox), YAML, JSON (JSON, MultiJson), TOML (Toml-rb, Tomlib), CSV | Lutaml::Model does not support CSV.
4+h| XML features
| <<root-namespace,XML default namespace>>
| Yes. Supports <root xmlns='http://example.com'> through the namespace option without prefix.
| No. Only supports <root xmlns:prefix='http://example.com'>.
|
| XML mixed content support | Yes. Supports the following kind of XML through <<mixed-content,mixed content>> support.
| No. Shale's map_content only supports the first text node.
|
| XML namespace inheritance
| Yes. Supports the <<namespace-inherit,inherit>> option to inherit the
namespace from the root element.
| No.
|
| Support for xsi:schemaLocation
| Yes. Automatically supports the <<xml-schema-location,xsi:schemaLocation>>
attribute for every element.
| Requires manual specification on every XML element that uses it.
|
| Compiling XML Schema to Lutaml::Model::Serializable classes
| Yes. Using <<xml-schema-to-model-files, Lutaml::Model::Schema#from_xml>>
simple types with restrictions and/or other validations.
|4+h| Attribute features
| Attribute delegation
| :delegate option to delegate attribute mappings to a model.
| :receiver option to delegate attribute mappings to a model.
|
| Enumerations
| Yes. Supports enumerations as value types through the
<<attribute-enumeration,values: option>>.
| No.
| Lutaml::Model supports enumerations as value types.
| Attribute extraction | Yes. Supports <<attribute-extraction,attribute extraction>> from key-value data models. | No. | Lutaml::Model supports attribute extraction from key-value data models.
|===
[[migrate-from-shale]] == Migration steps from Shale
The following sections provide a guide for migrating from Shale to Lutaml::Model.
=== Step 1: Replace inheritance class
Lutaml::Model uses Lutaml::Model::Serializable as the base inheritance class.
class Example < Lutaml::Model::Serializable
Lutaml::Model also supports an inclusion method as in the following example,
which is not supported by Shale. This is useful for cases where you want to
include the serialization methods in a class that already inherits from another
class.
class Example include Lutaml::Model::Serialize
====
Shale uses Shale::Mapper as the base inheritance class.
class Example < Shale::Mapper
Actions:
Shale::Mapper with Lutaml::Model::Serializable.=== Step 2: Replace value type definitions
Value types in Lutaml::Model are under the Lutaml::Model::Type module,
or use the LutaML type symbols.
Lutaml::Model supports specifying predefined value types as strings or
symbols, which is not supported by Shale.
====
Value types in Shale are under the Shale::Type module.
Action:
Shale::Type with Lutaml::Model::Type.=== Step 3: Configure serialization adapters
Lutaml::Model uses a configuration block to set the serialization adapters.
The equivalent for Shale is this:
Here are places that this code may reside at:
lib/your_gem_name.rb.spec/spec_helper.rb.Actions:
Replace the Shale configuration block with the Lutaml::Model::Config
configuration block.
Replace the Shale adapter with the Lutaml::Model adapter.
=== Step 4: Rewrite custom serialization methods
There is an implementation difference between Lutaml::Model and Shale for custom serialization methods.
Custom serialization methods in Lutaml::Model map to individual attributes.
For custom serialization methods, Lutaml::Model uses the :with keyword
instead of the :using keyword used by Shale.
class Example < Lutaml::Model::Serializable attribute :name, :string attribute :size, :integer attribute :color, :string attribute :description, :string
json do map "name", to: :name, with: { to: :name_to_json, from: :name_from_json } map "size", to: :size map "color", to: :color, with: { to: :color_to_json, from: :color_from_json } map "description", to: :description, with: { to: :description_to_json, from: :description_from_json } end
xml do root "CustomSerialization" map_element "Name", to: :name, with: { to: :name_to_xml, from: :name_from_xml } map_attribute "Size", to: :size map_element "Color", to: :color, with: { to: :color_to_xml, from: :color_from_xml } map_content to: :description, with: { to: :description_to_xml, from: :description_from_xml } end
def name_to_json(model, doc) doc["name"] = "JSON Masterpiece: #{model.name}" end
def name_from_json(model, value) model.name = value.sub(/^JSON Masterpiece: /, "") end
def color_to_json(model, doc) doc["color"] = model.color.upcase end
def color_from_json(model, value) model.color = value.downcase end
def description_to_json(model, doc) doc["description"] = "JSON Description: #{model.description}" end
def description_from_json(model, value) model.description = value.sub(/^JSON Description: /, "") end
def name_to_xml(model, parent, doc) el = doc.create_element("Name") doc.add_text(el, "XML Masterpiece: #{model.name}") doc.add_element(parent, el) end
def name_from_xml(model, value) model.name = value.sub(/^XML Masterpiece: /, "") end
def color_to_xml(model, parent, doc) color_element = doc.create_element("Color") doc.add_text(color_element, model.color.upcase) doc.add_element(parent, color_element) end
def color_from_xml(model, value) model.color = value.downcase end
def description_to_xml(model, parent, doc) doc.add_text(parent, "XML Description: #{model.description}") end
Custom serialization methods in Shale do not map to specific attributes, but allow the user to specify where the data goes.
class Example < Shale::Mapper attribute :name, Shale::Type::String attribute :size, Shale::Type::Integer attribute :color, Shale::Type::String attribute :description, Shale::Type::String
json do map "name", using: { from: :name_from_json, to: :name_to_json } map "size", to: :size map "color", using: { from: :color_from_json, to: :color_to_json } map "description", to: :description, using: { from: :description_from_json, to: :description_to_json } end
xml do root "CustomSerialization" map_element "Name", using: { from: :name_from_xml, to: :name_to_xml } map_attribute "Size", to: :size map_element "Color", using: { from: :color_from_xml, to: :color_to_xml } map_content to: :description, using: { from: :description_from_xml, to: :description_to_xml } end
def name_to_json(model, doc) doc['name'] = "JSON Masterpiece: #{model.name}" end
def name_from_json(model, value) model.name = value.sub(/^JSON Masterpiece: /, "") end
def color_to_json(model, doc) doc['color'] = model.color.upcase end
def color_from_json(model, doc) model.color = doc['color'].downcase end
def description_to_json(model, doc) doc['description'] = "JSON Description: #{model.description}" end
def description_from_json(model, doc) model.description = doc['description'].sub(/^JSON Description: /, "") end
def name_from_xml(model, node) model.name = node.text.sub(/^XML Masterpiece: /, "") end
NOTE: There are cases where the Shale implementation of custom methods work differently from the Lutaml::Model implementation. In these cases, you will need to adjust the custom methods accordingly.
Actions:
using keyword with the with keyword.== About LutaML
The name "LutaML" is pronounced as "Looh-tah-mel".
The name "LutaML" comes from the Latin word for clay, "Lutum", and "ML" for "Markup Language". Just as clay can be molded and modeled into beautiful and practical end products, the Lutaml::Model gem is used for data modeling, allowing you to shape and structure your data into useful forms.
== License and Copyright
This project is licensed under the BSD 2-clause License. See the link:LICENSE.md[] file for details.
Copyright Ribose.
FAQs
Unknown package
We found that lutaml-model demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Bybit's $1.46B hack by North Korea's Lazarus Group pushes 2025 crypto losses to $1.6B in just two months, already surpassing all of 2024's $1.49B total.
Security News
OpenSSF has published OSPS Baseline, an initiative designed to establish a minimum set of security-related best practices for open source software projects.
Security News
Michigan TypeScript founder Dimitri Mitropoulos implements WebAssembly runtime in TypeScript types, enabling Doom to run after processing 177 terabytes of type definitions.