azureml-fsspec

Filesystem interface to Azure Machine Learning defined URIs

Getting started

This package can be installed using:

pip install azureml-fsspec

Examples

Accepted uri format is Azure Machcine Learning defined datastore uri: azureml://subscriptions/([^/]+)/resourcegroups/([^/]+)/workspaces/([^/]+)/datastores/([^/]+)/paths/([^/]+)

# load parquet file to pandas
import pandas
df = pandas.read_parquet('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.parquet')

# load csv file to pandas
import pandas
df = pandas.read_csv('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.csv')

# load parquet file to dask
import dask.dataframe as dd
df = dd.read_parquet('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.parquet')

# load csv file to dask
import dask.dataframe as dd
df = dd.read_csv('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.csv')

Release History

1.3.1 (02-05-2024)

Features Added

• added support for using mlclient for authorization

1.3.0 (01-17-2024)

Features Added

• added registry support for fsspec.

1.2.0 (08-14-2023)

Features Added

• added auth dict support (SP and managedIdentity) for fsspec.
• added user exception mapping

1.1.0 (06-01-2023)

Features Added

• upload/download performance improvement

1.0.0 (04-07-2023)

Features Added

• Added upload/download API
• bugfix(root behavior)

0.1.0b1

Features Added

• Initial public preview release of FileSystem for Azure Machine Learning defined datastore uri

0.1.0b2

Features Added

• Fix release notes

0.1.0b3

Features Added

• Add mode in open API
• Add validation
• Upgrade dataprep to 4.8