mldesigner

Azure Machine Learning Designer Python SDK

The mldesigner package provide the SDK interface which work along with Azure ML Designer (drag-n-drop ML) UI experience.

• Azure ML Designer (drag-n-drop ML): designer is a UI tool in the Azure ML workspace for visually connecting datasets and components on an interactive canvas to create machine learning pipelines. To learn how to get started with the designer, see Tutorial: Predict automobile price with the designer.

Especially, the package ease the authoring experience of resources like Components & Pipelines:

• Components: self-contained piece of code that does one step in a machine learning pipeline: data preprocessing, model training, model scoring, a hyperparameter tuning run, etc. Such that it can be parameterized and then used in different contexts.
• Pipelines: independently executable workflow of machine learning tasks composed by Components.

Change Log

v0.1.0b18 (2024.01.31)

Recommended to work with azure-ai-ml==1.13

New Features

• mldesigner compile supports to compile parallel component.
• mldesigner generate supports to generate package with provided authentication.

v0.1.0b17 (2023.11.14)

Recommended to work with azure-ai-ml==1.12

Improvements

• [Compile] Update compiled command component schema path to https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json.

Fixed Bugs

• [Compile] Fixed the error when source component inputs is empty.

v0.1.0b16 (2023.10.18)

Recommended to work with azure-ai-ml==1.11

New Features

• mldesigner compile supports to compile with .amlignore:
  • mldesigner compile --source xxx.yaml --amlignore-file <path-to>/.amlignore

v0.1.0b15 (2023.09.13)

Recommended to work with azure-ai-ml==1.10

New Features

• mldesigner compile support v1.5 spark components.
• is_control=True is not required for primitive type output.
• generate package support specify version for registry component and generate by sdk support remote url.

  from mldesigner import generate
  generate(
    source="azureml://registries/{registry_name}/components/{component_name}/versions/0.0.8",
  )
  

v0.1.0b14 (2023.07.31)

Recommended to work with azure-ai-ml==1.9.0

New Features

• Support primitive type as component return annotation and not need to write is_control=True.
• Support Annotated type as component input or component return annotation.
• Support Annotated type in @group.

  from mldesigner import command_component, Output, Meta
  from typing_extensions import Annotated
  
  @command_component()
  def my_component1(input_int: int) -> int:
      return input_int
  
  @command_component()
  def my_component2(input_int: Annotated[int, Meta(description="test annotation int", min=1, max=10, default=1)],) \
                                                       -> Annotated[int, Meta(description="test annotation int")]:  # using Meta class to configure more meta data
      return input_int
  

v0.1.0b13 (2023.05.16)

Recommended to work with azure-ai-ml==1.7.0

Improvements

• Support to generate components with specified version or label:
  • azureml://subscriptions//resourcegroups/<resource_group>/workspaces//components/<component_name>/labels/
  • azureml://subscriptions//resourcegroups/<resource_group>/workspaces//components/<component_name>/versions/
• Refine the log content of generate component package.

v0.1.0b12 (2023.03.28)

Recommended to work with azure-ai-ml==1.5.0

Improvements

• Support multiple types of identity for dynamic pipeline creation, the priority is user identity > managed identity > others.
• Refine mldesigner execute:
  • When component name is not specified, if there is only one component in the source file, execute this component, raise error when there are multiple components.
  • If required parameters are not provided, list all of them and raise error.
• Improve support for enum values in generate_package.
  • Support enum values with no \w characters, e.g. empty string, -, + and \t.
  • Support enum values with duplicate sanitized names, e.g. A and a.
• Use curated environment as default mldesigner component environment:
  • For command component: azureml://registries/azureml/environments/mldesigner-minimal/labels/latest
  • For dynamic pipeline: azureml://registries/azureml/environments/mldesigner/labels/latest
• Support mltable type in @dynamic outputs

Fixed Bugs

• Fix wrong command error when mldesigner component code is set to be a file instead of a folder.

v0.1.0b11 (2023.02.09)

Recommended to work with azure-ai-ml==1.4.0

Improvements

• Add python version requirements <4.0,>=3.7

v0.1.0b10 (2023.01.20)

Recommended to work with azure-ai-ml==1.3.0

New Features

• Support primitive type as component return annotation.
• Support @group as component return annotation.
• Add validation for component execution results according to return annotation.

Improvements

• Return component execution result as mldesigner execute result.

Fixed Bugs

• Fixed missing keyword arguments when executing component in an environment that has both mldesigner and azure-ai-ml
• Fixed compile error when compiling pipeline that has a node uses remote registered component either from workspace or registry.
• Fixed parse error when input string is a json serialized string.

v0.1.0b9 (2022.12.06)

Recommended to work with azure-ai-ml==1.2.0

Improvements

• Enable creating component with auto-incremented version if no version specified.
• Update mldesigner default environment image to be mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04
• Mldesigner compile improvements:
  • Exits with code 1 when there is failed compilation.
  • Output yaml file with original file name if the input is a yaml.
  • Generated pipeline node keys is ordered.

Fixed Bugs

• Fixed mldesigner executor does not pass **kwargs to component func

v0.1.0b8 (2022.10.26)

Fixed Bugs

• Fixed dependency issue when importing typing_extensions

v0.1.0b7 (2022.10.25)

New Features

• Support mldesigner compile: mldesigner compile
• Support list type component input for generate_package.

Fixed Bugs

• Fixed execute error when input string has space or "=" inside.
• Fixed execute error for bool parameter parsing.

Improvements

• Remove mode client default value ro_mount/rw_mount from mldesigner Input/Output class.
• Refine error message when defining a component in .ipynb.
• Refine error message when failed to create component entity.

v0.1.0b6 (2022.09.19)

Improvements:

• Adopted optional input new format: '$[[]]' instead of old '[]'.
• Support Enum for mldesigner input.
• Raise error if input has no annotation specified.

Fixed Bugs:

• Fixed import error when used along with azure-ai-ml<0.1.0b7.

v0.1.0b5 (2022.09.08)

New Features

• Enable using @command_component without brackets when no additional parameters.

  from mldesigner import command_component
  @command_component
  def my_component():
      pass
  
  # equals to
  @command_component()
  def my_component():
      pass
  

Fixed Bugs:

• Fixed component command execution error when no inputs or outputs specified.
• Fixed incompatible issue with azure-ai-ml >=0.1.0b7: error when trying to import azure-ai-ml constants

v0.1.0b4 (2022.08.22)

New Features:

• Support mldesigner generate:
  • Generate component package from local yaml files or remote source
    • See reference doc: mldesigner generate doc
    • Sample notebook: mldesigner generate samples
• Support mldesigner execute: execute component in local host environment.
  • CLI example: mldesigner execute --source ./components.py --name my_component inputs a=1 b=2
  • SDK example:

    from mldesigner import execute
    from components import my_component
    node = my_component(a=1, b=2)
    res = execute(node)
    

  • More information: mldesigner execute --help

Improvements:

• Fix bump version config.

v0.1.0b3 (2022.07.14)

New Features:

• Support optional input for mldesigner Input class.

  from mldesigner import command_component, Input
  @command_component()
  def my_component(optional_param: Input(type="integer", optional=True)):
      pass
  

• Support io descriptions inferring from docstring for pipeline & component.

Improvements:

• Handle compatibility issue for future changes. Less imports of private functions
• Remove default property for mldesigner Input class.
• Enable mldesigner to use argparser to parse incoming args.
• Compatibility handling: old azure ai ml package use different way to load component.
• Component input like "int_param=3", no longer to be processed as optional input when registering to remote.

v0.1.0b2 (2022.05.23)

Improvements:

• Refine code terminologies, replace old dsl with new mldesigner.

v0.1.0b1 (2022.05.20)

New Features:

• Support using decorator @command_component to define a component.
  • Create a component:

    from mldesigner import command_component, Input, Output
    
    @command_component()
    def hello_world(input: Input, output: Output, param='str_param'):
        print("Hello World!")
    

  • Register the component to server:

    from azure.ai.ml import MLClient
    client = MLClient.from_config(credential=credential)
    client.components.create_or_update(hello_world)
    

  • Use sdk component in pipeline:

    from azure.ai.ml import dsl
    
    @dsl.pipeline()
    def my_pipeline():
        node = hello_world()
        return {"pipeline_output": node.outputs.output}
    
    pipeline = my_pipeline()
    client.jobs.create_or_update(pipeline)