update examples for CLI (v2) public preview refresh (Azure#890)

* update setup to use preview CLI * Create setup.sh * update schemas to preview * Update setup.sh * Update setup.sh * Update setup.sh * Update setup.sh * Update setup.sh * add environment examples (Azure#736) * Create docker image example * Rename docker image example to docker_image.yml * Create Dockerfile * Create docker_context.yml * refactor and updates * update creds, readme, workflows * fix * remove workflow * fix mistake * fix Co-authored-by: Cody <[email protected]> * Update docker-context.yml (Azure#739) * Update setup.sh * Update setup.sh * Update setup.sh * Update setup.sh * major jobs refactor; add pipeline job (Azure#745) * add basic and nyc taxi pipelines samples under jobs * major refactor * minor edit * minor edit * save * remove sweep dir * update inputs outputs and compute for cli 0.7 * update component yamls nyc taxi example Co-authored-by: Sharmeelee Bijlani <[email protected]> * Create simple-compute-cluster.yml (Azure#748) * update compute cluster examples (Azure#752) * update compute examples * shorten name * low_priorty -> lowpriority * add locaiton example * add compute-instance * remove instances from testing * refactor jobs; add simple pipeline examples (Azure#753) * refactor jobs * save * pipeline edits * switch to canary * switch to eastus * fully test * switch back to master * reduce python-sdk testing freq; cli and setup cleanup (Azure#759) * repo health * rerun readme * environment example updates (Azure#760) * remove version pinning; add version pinning * edits * Update setup.sh * add workflow_dispatch: to cli workflows (Azure#761) * touch setup.sh * add workflow_dispatch: to cli jobs * Update setup.sh * Update setup.sh * fix low priority compute example (Azure#762) * Update setup.sh * public preview refresh changes (Azure#750) * fixes * changes * revert setup.sh changes * latest cli * cli changes * update dir structure * update name * update cli version * cli version * cli version * cleanup * update comments Co-authored-by: Cody <[email protected]> * Update setup.sh * drop _job (Azure#772) * Add files via upload * fix validation scripts for user specified identity and system assigned identity (Azure#747) * fix sai + uai scripts * remove set x * fix workspace name * fix arg * fix rest * add sku * change credentials to mir * touch * add check * add extra line ending * add object info to role assignment * forgotten vars * fix role assignment * fix other role assignments * use latest version of sdk * fix arm calls * remove hardcoded vals * fix hardcoded values * replace endpoint name with var * add euap location * remove pined version of cli * fix sai role assignment * update credential logic in readme for uai, sai * format code * fixed comments * cleaned up * cleanup * revert * fix tags * fix uai Co-authored-by: max <[email protected]> Co-authored-by: Cody <[email protected]> * clean up nyc taxi pipeline (Azure#773) * update sweep timeout (Azure#778) * pin badges to main branch (Azure#779) * pin badges to main branch * save * switch testing to master * manually update schemas * Update setup.sh * Update PyTorch example (Azure#791) * Update setup.sh * add cifar-10 pipeline example (Azure#794) * save * save - training updates * remove cifar from untested jobs * save * save * switch setup to eastus * set CR to false * fix some jobs * update readme * save * save * fix setup * run readme * add data copy script * edit * edit * save * save * fix * save * edit cifar loc * fix * fix * Update setup.sh * batch GA samples (Azure#722) * batch score GA samples * update batch * update batch * update batch sample * update anon env * move non mlflow up * update test flow * clean up batch sample * update batch sample * update schema ref * rename yml files * update yml file name * minor update * Update setup.sh * Update workspace CLI examples (Azure#767) * Update workspace CLI examples * updating examples per PR feedback * property ordering and update app insights attr fix * include example on adding private link endpoint * include private link example * simplify ple example * disabled in capital and consistent resource suffix * consisten resource names * save * fix * add ws delete * fixes * fix * fix * remove workspace script Co-authored-by: Cody <[email protected]> * add diabetes data * fix * fix * save * Fixed typo in snippet (Azure#796) * pipelines-with-components basic examples (Azure#777) * pipelines-with-components * data samples * tf hello world * create components and dataset in repo setup * pytorch hello world * update readme; format code * update create-datasets.sh * fix readme; remove unused job; run readme * add missing files * setup updates Co-authored-by: Cody <[email protected]> * create variables (Azure#799) * local endpoint test script (Azure#804) * script for local endpoint * local endpoint * Added Dataset and Datastore YAML samples. (Azure#800) * Added Dataset and Datastore YAML samples. * Updated README files. * renames and minor updates * Updated Dataset samples README. Co-authored-by: Cody <[email protected]> * standardization and fixes (Azure#805) * update schemas * update assets * update resources * update setup-repo * save * udpate assets * update run logic (Azure#806) * update schemas * update assets * update resources * update setup-repo * save * udpate assets * update run logic * omit run-job.sh * print job uri * remove workflow * edit run-job.sh * update run-job.sh * update run-job.sh * fix cifar pipeline (Azure#809) * Update setup.sh * ; => : (Azure#812) * model-dir => model_dir (Azure#813) * update run-job.sh * mount => ro_mount (Azure#814) * fix nyc-taxi pipeline example (Azure#815) * ever feel stupid? I feel stupid * fix 4d_dataset_input (Azure#816) * clean up batch endpoints (Azure#817) * switch RG (Azure#818) * Update setup.sh * format code * update jobs and things (Azure#822) * save * save * save * save * save * save * save * save * format code * remove cli/jobs/basics/data * save * save * save * update * fix * minor edit * edit * switch to eastus * format code * Add functional Batch Endpoint REST sample (Azure#820) * add batch score rest sample * add git workflow * update REST sample * update sample * Functional October batch REST sample -- score from Dataset failing on request * Clean up script * run readme * Fix some quoting issues * reset location * Fix modelUri * Fail early if job submit failed * Reset readme.py * Fix location to be based on workspace * Add other terminal state Co-authored-by: Tracy Chen <[email protected]> Co-authored-by: Cody <[email protected]> * minor edits * save * update * job updates (Azure#824) * save * updates * minor updates * minor update * fix cifar dataset * Update hello-iris-file.yml * fixes (Azure#827) * fixes * fix * fix * add docker context job example for sklearn (Azure#828) * add docker context job example for sklearn * run readme * change image * Add missing section opener (Azure#829) Missing opening comment for create_environment * minor edit * remove arc examples (Azure#830) * save * add hello-code * autoscale (Azure#832) * fix unique output (Azure#831) * fix unique output * fix unique output * beautify json * update REST invoke * fix typo * remove delete prompt * unique output folder * add deployment delete * consistency edits (Azure#834) * --set experiment_name in train.sh (Azure#835) * fix instance (Azure#836) * minor consistency edits (Azure#837) * removing aks sample from cli-preview (Azure#844) * Update torchserve BYOC sample (Azure#845) * add notebooks via papermill job examples (Azure#847) * add notebooks via papermill job examples * output notebook to ./outputs/ * fix tag (Azure#848) * byoc tfserving sample (Azure#853) * byoc tfserving sample * deployment name fix * r-byoc-update (Azure#854) * Update setup.sh * update * update * update vm attach example (Azure#855) * Revert "update vm attach example" (Azure#856) This reverts commit f3a7efe. * add env version for component examples (Azure#858) * fix typo in directory; rw-mount to rw_mount (Azure#859) * fix spark example (Azure#857) * fix spark example * save * try pin pip * ? * try older pip * pin image * format code * update rest sample (Azure#861) * update rest sample * fix format * fix typo * fix output path * update outputdataset to output * update output * move dask to named output * add tag for wait (Azure#866) * Fixing closing region tag (Azure#867) read_condafile section tags were not correctly closed. * swap install steps in setup.sh * edit comment * run black * remove CIs from refresh testing (Azure#873) * add defaults list to misc script (Azure#874) * https => wasbs for dask folder example (Azure#875) * https => wasbs for dask folder example * test moving container * Adding MLflow sample for online endpoints (Azure#872) * Adding MLflow sample for online endpoints * run readme.py * run black * Revert "run black" This reverts commit b649bb5. * updating sh file * Adding MLflow model deployment sample * Updating files to be consistent * updating .sh file * updating deployment YAML and .sh file * adding debug flag for model delete * updating YAML and .sh script * updating .sh script Co-authored-by: Cody <[email protected]> * Update triton samples to ncd - single model (Azure#876) * Update triton samples to ncd - single model * minor spelling error * addressing build failures -1 * addressing build failures -2 * addressing missing dependencies * adding missing dependency 2 * update pip install tritonclient[http] * addressing build failures * minor update to tags (Azure#877) * remove extra triton examples (Azure#881) * remove flaml notebook action * add ssh settings to vm attach (Azure#883) * remove .schemas; consistency edit to scripts (Azure#887) * remove .schemas * consistency edit * remove unused workflow * switch cli install to public * remove broken job (Azure#892) * fix merge issues * fix job logic Co-authored-by: Saachi Gopal <[email protected]> Co-authored-by: Sharmeelee Bijlani <[email protected]> Co-authored-by: vijetajo <[email protected]> Co-authored-by: Sethu Raman <[email protected]> Co-authored-by: Bala P V <[email protected]> Co-authored-by: max markin <[email protected]> Co-authored-by: max <[email protected]> Co-authored-by: Paul Shealy <[email protected]> Co-authored-by: Tracy Chen <[email protected]> Co-authored-by: Dennis Eikelenboom <[email protected]> Co-authored-by: Larry O'Brien <[email protected]> Co-authored-by: Manoj Bableshwar <[email protected]> Co-authored-by: Yogi Pandey <[email protected]> Co-authored-by: Christopher Pirillo <[email protected]> Co-authored-by: Larry Franks <[email protected]> Co-authored-by: Shivani Santosh Sambare <[email protected]> Co-authored-by: Daniel Schneider <[email protected]>
savitamittal1 · Nov 4, 2021 · 218cc4c · 218cc4c
1 parent 13f2e1b
commit 218cc4c
Show file tree

Hide file tree

Showing 607 changed files with 30,304 additions and 18,419 deletions.
diff --git a/...ssets-environment-python-ml-basic-cpu.yml → ...s/cli-assets-dataset-cloud-file-https.yml b/...ssets-environment-python-ml-basic-cpu.yml → ...s/cli-assets-dataset-cloud-file-https.yml
@@ -1,5 +1,6 @@
-name: cli-assets-environment-python-ml-basic-cpu
+name: cli-assets-dataset-cloud-file-https
 on:
+  workflow_dispatch:
   schedule:
     - cron: "0 0/4 * * *"
   pull_request:
@@ -8,8 +9,8 @@ on:
       - cli-preview
       - releases/current
     paths:
-      - cli/assets/environment/python-ml-basic-cpu.yml
-      - .github/workflows/cli-assets-environment-python-ml-basic-cpu.yml
+      - cli/assets/dataset/cloud-file-https.yml
+      - .github/workflows/cli-assets-dataset-cloud-file-https.yml
       - cli/setup.sh
 jobs:
   build:
@@ -20,11 +21,11 @@ jobs:
     - name: azure login
       uses: azure/login@v1
       with:
-        creds: ${{secrets.AZ_AE_CREDS}}
+        creds: ${{secrets.AZ_CREDS}}
     - name: setup
       run: bash setup.sh
       working-directory: cli
       continue-on-error: true
     - name: create asset
-      run: az ml environment create -f assets/environment/python-ml-basic-cpu.yml
+      run: az ml dataset create -f assets/dataset/cloud-file-https.yml
       working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-cloud-file-wasbs.yml b/.github/workflows/cli-assets-dataset-cloud-file-wasbs.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-cloud-file-wasbs
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/cloud-file-wasbs.yml
+      - .github/workflows/cli-assets-dataset-cloud-file-wasbs.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/cloud-file-wasbs.yml
+      working-directory: cli
diff --git a/...kflows/cli-assets-model-lightgbm-iris.yml → ...rkflows/cli-assets-dataset-cloud-file.yml b/...kflows/cli-assets-model-lightgbm-iris.yml → ...rkflows/cli-assets-dataset-cloud-file.yml
@@ -1,5 +1,6 @@
-name: cli-assets-model-lightgbm-iris
+name: cli-assets-dataset-cloud-file
 on:
+  workflow_dispatch:
   schedule:
     - cron: "0 0/4 * * *"
   pull_request:
@@ -8,8 +9,8 @@ on:
       - cli-preview
       - releases/current
     paths:
-      - cli/assets/model/lightgbm-iris.yml
-      - .github/workflows/cli-assets-model-lightgbm-iris.yml
+      - cli/assets/dataset/cloud-file.yml
+      - .github/workflows/cli-assets-dataset-cloud-file.yml
       - cli/setup.sh
 jobs:
   build:
@@ -20,11 +21,11 @@ jobs:
     - name: azure login
       uses: azure/login@v1
       with:
-        creds: ${{secrets.AZ_AE_CREDS}}
+        creds: ${{secrets.AZ_CREDS}}
     - name: setup
       run: bash setup.sh
       working-directory: cli
       continue-on-error: true
     - name: create asset
-      run: az ml model create -f assets/model/lightgbm-iris.yml
+      run: az ml dataset create -f assets/dataset/cloud-file.yml
       working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-cloud-folder-https.yml b/.github/workflows/cli-assets-dataset-cloud-folder-https.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-cloud-folder-https
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/cloud-folder-https.yml
+      - .github/workflows/cli-assets-dataset-cloud-folder-https.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/cloud-folder-https.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-cloud-folder-wasbs.yml b/.github/workflows/cli-assets-dataset-cloud-folder-wasbs.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-cloud-folder-wasbs
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/cloud-folder-wasbs.yml
+      - .github/workflows/cli-assets-dataset-cloud-folder-wasbs.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/cloud-folder-wasbs.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-cloud-folder.yml b/.github/workflows/cli-assets-dataset-cloud-folder.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-cloud-folder
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/cloud-folder.yml
+      - .github/workflows/cli-assets-dataset-cloud-folder.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/cloud-folder.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-iris-csv-example.yml b/.github/workflows/cli-assets-dataset-iris-csv-example.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-iris-csv-example
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/iris-csv-example.yml
+      - .github/workflows/cli-assets-dataset-iris-csv-example.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/iris-csv-example.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-local-file.yml b/.github/workflows/cli-assets-dataset-local-file.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-local-file
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/local-file.yml
+      - .github/workflows/cli-assets-dataset-local-file.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/local-file.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-local-folder.yml b/.github/workflows/cli-assets-dataset-local-folder.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-local-folder
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/local-folder.yml
+      - .github/workflows/cli-assets-dataset-local-folder.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/local-folder.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-dataset-public-file-https.yml b/.github/workflows/cli-assets-dataset-public-file-https.yml
@@ -0,0 +1,31 @@
+name: cli-assets-dataset-public-file-https
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/dataset/public-file-https.yml
+      - .github/workflows/cli-assets-dataset-public-file-https.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml dataset create -f assets/dataset/public-file-https.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-environment-docker-context.yml b/.github/workflows/cli-assets-environment-docker-context.yml
@@ -0,0 +1,31 @@
+name: cli-assets-environment-docker-context
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/environment/docker-context.yml
+      - .github/workflows/cli-assets-environment-docker-context.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml environment create -f assets/environment/docker-context.yml
+      working-directory: cli
diff --git a/.github/workflows/cli-assets-environment-docker-image-plus-conda.yml b/.github/workflows/cli-assets-environment-docker-image-plus-conda.yml
@@ -0,0 +1,31 @@
+name: cli-assets-environment-docker-image-plus-conda
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0/4 * * *"
+  pull_request:
+    branches:
+      - main
+      - cli-preview
+      - releases/current
+    paths:
+      - cli/assets/environment/docker-image-plus-conda.yml
+      - .github/workflows/cli-assets-environment-docker-image-plus-conda.yml
+      - cli/setup.sh
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: check out repo
+      uses: actions/checkout@v2
+    - name: azure login
+      uses: azure/login@v1
+      with:
+        creds: ${{secrets.AZ_CREDS}}
+    - name: setup
+      run: bash setup.sh
+      working-directory: cli
+      continue-on-error: true
+    - name: create asset
+      run: az ml environment create -f assets/environment/docker-image-plus-conda.yml
+      working-directory: cli