Assets

Creating an asset

Examples of various asset types are located throughout this repository. Some specific places to look for specific asset types are:

• Components
• Data: TBD
• Environments
• Models

Asset names

Asset names matter! Please refer to the Asset naming convention for the details.

Asset files

Go to your team's directory in the repo and create a new directory for your asset with these contents:

my_env/               # Directory name should match asset name
├── asset.yaml        # Asset config file
├── spec.yaml         # Spec file for use with Azure CLI
│                     # And maybe some additional files; see below

Components will have the following additional files:

└── src/           # Component source directory, can be shared with other components
    └── script.py  # Script that controls the component

Environments will have the following additional files:

├── environment.yaml  # Environment config file
└── context/          # Docker build context (may contain additional files)
    └── Dockerfile    # Dockerfile

Models will have the following additional files:

└── model.yaml  # Model config file

spec.yaml

An Azure CLI v2 spec file. Content will vary depending on asset type.

Template tags

Spec files associated with assets in this repository may contain the following template tags:

Tag	Resolves to
{{asset.name}}	Name of the asset, from the asset config
{{asset.version}}	Version of the asset, from the asset config
{{asset.repo.commit_hash}}	Git commit hash of the commit being released
{{asset.repo.url}}	Git URL for this repository

asset.yaml

The asset config file provides configuration that is common to all types of assets.

name: my_env                    # Asset name. If omitted, will be read from spec's name property.
version: auto                   # Asset version. If omitted, will be read from spec's version property. "auto" means automatic 1-up versioning, but currently available only for environments.
type: environment               # Asset type. Must be data, component, environment, or model.
spec: spec.yaml                 # Spec file, relative to asset config file
extra_config: environment.yaml  # Environment or model config file, relative to asset config file
release_paths:                  # Optional, used to include additional dirs/files during release
- ../src                        # Paths are relative to asset config file
- "!../src/test"                # Exclude dirs/files by using a ! prefix
test:                           # Optional, used to enable pytest testing
  pytest:                                     # Config items for pytest 
    enabled: true                             # Flag to enable/disable testing
    pip_requirements: tests/requirements.txt  # Requirements installed before testing
    tests_dir: tests                          # Directory containing tests
categories:                                   # Categories for the asset, used when generating documentation
- PyTorch
- "Training/GPU"                              # Subcategories can be specified, separated by slashes

Categories

Categories provided in the asset.yaml file can help customers find assets that are relevant to their needs. When documentation is generated, assets are first categorized by their asset type (component, environment, etc.) and then by any categories listed in their asset.yaml file. Because of this, an asset's type should not be included in its category list. Multiple categories are allowed, and each category may contain subcategories (see example above). When choosing your own categories, it may help to refer to the generated reference documentation to see how other teams are using them.

Releasing an asset

The assets-release workflow runs each time a PR merges to the main branch. The workflow performs the following actions:

1. Searches the main branch for asset.yaml files
2. Compares each asset against the contents of release branch to identify whether it's new, updated, or unchanged
3. Adds new assets to the release branch
4. Updates changed assets, if one of the following is true:
   • The asset is auto versioned
   • It's manually versioned and hasn't been released yet
5. Removes assets no longer in the main branch from the release branch
6. Commits changes to the release branch

Assets are released as needed from the contents of the release branch. As part of the release process an <asset_type>/<name>/<version> tag is added to the repo for reference. These steps occur outside of GitHub and involve steps like building and publishing Docker images (for environments), and then creating the asset in the azureml registry.

Deprecating assets

An asset can be deprecated when it is no longer needed. This is done by:

• Replacing the description for all versions of the asset with: DEPRECATED: Please use <recommended replacement asset> instead.
• Adding a Deprecated tag with an empty value
• Archiving all versions of the asset
• Removing it from azureml-asset's main branch