components documentation

Components

Categories

• AOAI

• AutoML

• AutoML Image training

• Basic

• Batch Score

• Benchmarking

• Demand Forecasting

• Deployment

• Distillation

• Feature Store

• Finetune

• Foundational Models

• Foundational Models Finetune

• High

• Low

• Medium

• Model

• Model Evaluation

• Model Monitoring

• Models

• NLP Multiclass

• NLP Multilabel

• NLP NER

• OAI

• Premium

• Responsible AI

• Retrieval Augmented Generation

• Sample

• Singularity

• Standard

All components

---

• action_analyzer_correlation_test

  Perform correlation test on different groups to generate actions.

• action_analyzer_identify_problem_traffic

  Separate bad queries into different groups.

• action_analyzer_metrics_calculation

  Calculate futher metrics for generating actions.

• action_analyzer_output_actions

  Merge and output actions.

• aoai_finetune_pipeline

  Pipeline component for proxy fine-tuning with AOAI

• aoai_finetuning

  Upload data to Azure OpenAI resource, finetune model and delete data

• automl_classification

  Component that kicks off an AutoML job to train a classification model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_forecasting

  Component that kicks off an AutoML job to train a forecasting model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_forecasting_inference

  Inference component for AutoML Forecasting.

• automl_hts_automl_training_step

• automl_hts_data_aggregation_step

• automl_hts_inference

  Enables inference for hts components.

• automl_hts_inference_collect_step

• automl_hts_inference_setup_step

• automl_hts_prs_inference_step

• automl_hts_training

  Enables AutoML Training for hts components.

• automl_hts_training_collect_step

• automl_hts_training_setup_step

• automl_image_classification

  Component that kicks off an AutoML job to train an image classification model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_image_classification_multilabel

  Component that kicks off an AutoML job to train an multilabel image classification model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_image_instance_segmentation

  Component that kicks off an AutoML job to train an image instance segmentation model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_image_object_detection

  Component that kicks off an AutoML job to train an image object detection model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_many_models_inference

  Inference components for AutoML many model.

• automl_many_models_inference_collect_step

• automl_many_models_inference_setup_step

• automl_many_models_inference_step

• automl_many_models_training

  Enables AutoML many models training.

• automl_many_models_training_collection_step

• automl_many_models_training_setup_step

• automl_many_models_training_step

• automl_regression

  Component that kicks off an AutoML job to train a regression model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_tabular_data_partitioning

  Enables dataset partitioning for AutoML many models and hierarchical timeseries solution accelerators using spark.

• automl_text_classification

  Component that kicks off an AutoML job to train a NLP text classification model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_text_classification_multilabel

  Component that kicks off an AutoML job to train a NLP text classification multilabel model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• automl_text_ner

  Component that kicks off an AutoML job to train a NLP NE (Named Entity Recognition) model within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• batch_benchmark_config_generator

  Generates the config for the batch score component.

• batch_benchmark_inference

  Components for batch endpoint inference

• batch_benchmark_inference_claude

  Components for batch endpoint inference

• batch_benchmark_inference_with_inference_compute

  Components for batch endpoint inference with inference compute support.

• batch_benchmark_score

• batch_deploy_model

  Batch deploy a model to a workspace. The component works on compute with MSI attached.

• batch_inference_preparer

  Prepare the jsonl file and endpoint for batch inference component.

• batch_output_formatter

  Output Formatter for batch inference output

• batch_resource_manager

  Resource Manager for batch inference.

• batch_score_llm

• batch_score_oss

• benchmark_embedding_model

  Component for benchmarking an embedding model via MTEB.

• benchmark_result_aggregator

  Aggregate quality metrics, performance metrics and all of the metadata from the pipeline. Also add them to the root run.

• chat_completion_datapreprocess

  Component to preprocess data for chat completion task. See docs to learn more.

• chat_completion_finetune

  Component to finetune Hugging Face pretrained models for chat completion task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• chat_completion_model_import

  Component to import PyTorch / MLFlow model. See docs to learn more.

• chat_completion_pipeline

  Pipeline Component to finetune Hugging Face pretrained models for chat completion task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• compute_metrics

  Calculate model performance metrics, given ground truth and prediction data.

• compute_performance_metrics

  Performs performance metric post processing using data from a model inference run.

• convert_model_to_mlflow

  Component converts models from supported frameworks to MLflow model packaging format

• data_delete

  Delete data file from Azure OpenAI resource

• data_drift_compute_metrics

  Compute data drift metrics given a baseline and a deployment's model data input.

• data_drift_signal_monitor

  Computes the data drift between a baseline and production data assets.

• data_quality_compute_metrics

  Compute data quality metrics leveraged by the data quality monitor.

• data_quality_data_statistics

  Compute data statistics leveraged by the data quality monitor.

• data_quality_metrics_joiner

  Join baseline and target data quality metrics into a single output.

• data_quality_signal_monitor

  Computes the data quality of a target dataset with reference to a baseline.

• data_upload

  Component to upload user's data from AzureML workspace to Azure OpenAI resource

• dataset_downloader

  Downloads the dataset onto blob store.

• dataset_preprocessor

  Dataset Preprocessor

• dataset_sampler

  Samples a dataset containing JSONL file(s).

• delete_endpoint

  Deletes an endpoint resource.

• deploy_model

  Deploy a model to a workspace. The component works on compute with MSI attached.

• diffusers_text_to_image_dreambooth_pipeline

  Pipeline component for text to image dreambooth training using diffusers library and transformers models.

• diffusers_text_to_image_finetune

  Component to finetune stable diffusion models using diffusers for text to image.

• diffusers_text_to_image_model_import

  Import PyTorch / MLflow model

• download_model

  Downloads a publicly available model

• evaluate_model

  Evaluate MLFlow models for supported task types.

• export_data_database

  Component that export data from uri_file data asset to database within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• feature_attribution_drift_compute_metrics

  Feature attribution drift using model monitoring.

• feature_attribution_drift_signal_monitor

  Computes the feature attribution between a baseline and production data assets.

• feature_importance_metrics

  Feature importance for model monitoring.

• feature_retrieval

  Retrieval component to be used to retrieve offline features from feature store.

• finetune_common_validation

  Component to validate the finetune job against Validation Service

• finetune_submit

  Component to submit FT job to Azure OpenAI resource

• ft_nlp_common_validation

  Component to validate the finetune job against Validation Service

• ft_nlp_model_converter

  Component to convert the finetune job output to pytorch and mlflow model

• genai_mdc_preprocessor

  Filters the raw span log based on the window provided, and aggregates it to trace level.

• genai_token_statistics_compute_metrics

  Compute token statistics metrics.

• genai_token_statistics_signal_monitor

  Computes the token and cost metrics over LLM outputs.

• generation_safety_quality_signal_monitor

  Computes the content generation safety metrics over LLM outputs.

• gsq_annotation_compute_histogram

  Compute annotation histogram given a deployment's model data input.

• gsq_annotation_compute_metrics

  Compute annotation metrics given a deployment's model data input.

• gsq_input_schema_adaptor

  Adapt data to fit into GSQ component.

• hello_command

  Command Component that takes in a string input message and prints it out.

• hello_pipeline

  Pipeline Component that takes in a string input message and passes it to the Hello World Command Component to be printed out.

• image_classification_pipeline

  Pipeline component for image classification.

• image_framework_selector

  Framework selector control flow component for image tasks

• image_instance_segmentation_pipeline

  Pipeline component for image instance segmentation.

• image_model_output_selector

  Model output selector control flow component for image tasks

• image_object_detection_pipeline

  Pipeline component for image object detection.

• import_data_database

  Component that import data from database as mltable data asset within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• import_data_file_system

  Component that import data from external file_system as uri_folder data asset within an Azure Machine Learning pipeline. For more details, you can look at the component documentation here (Preview).

• import_model

  Import a model into a workspace or a registry

• inference_postprocessor

  Inference Postprocessor

• llm_dbcopilot_create_promptflow

• llm_dbcopilot_deploy_endpoint

• llm_dbcopilot_grounding

• llm_dbcopilot_grounding_ground_samples

• llm_ingest_dataset_to_acs_basic

  Single job pipeline to chunk data from AzureML data asset, and create ACS embeddings index

• llm_ingest_dataset_to_acs_user_id

  Single job pipeline to chunk data from AzureML data asset, and create ACS embeddings index

• llm_ingest_dataset_to_faiss_basic

  Single job pipeline to chunk data from AzureML data asset, and create FAISS embeddings index

• llm_ingest_dataset_to_faiss_user_id

  Single job pipeline to chunk data from AzureML data asset, and create FAISS embeddings index

• llm_ingest_db_to_acs

  Single job pipeline to chunk data from AzureML sql data store, and create ACS embeddings index

• llm_ingest_db_to_faiss

  Single job pipeline to chunk data from AzureML sql data store, and create FAISS embeddings index

• llm_ingest_dbcopilot_acs_e2e

  Single job pipeline to chunk data from AzureML DB Datastore and create acs embeddings index

• llm_ingest_dbcopilot_faiss_e2e

  Single job pipeline to chunk data from AzureML DB Datastore and create faiss embeddings index

• llm_rag_crack_and_chunk

  Creates chunks no larger than chunk_size from input_data, extracted document titles are prepended to each chunk

LLM models have token limits for the prompts passed to them, this is a limiting factor at embedding time and even more limiting at prompt completion time as only so much context ca...

• llm_rag_crack_and_chunk_and_embed

  Creates chunks no larger than chunk_size from input_data, extracted document titles are prepended to each chunk

LLM models have token limits for the prompts passed to them, this is a limiting factor at embedding time and even more limiting at prompt completion time as only so much context ca...

• llm_rag_crack_chunk_embed_index_and_register

  Creates chunks no larger than chunk_size from input_data, extracted document titles are prepended to each chunk\n\n

LLM models have token limits for the prompts passed to them, this is a limiting factor at embedding time and even more limiting at prompt completion time as only so much contex...

• llm_rag_crawl_url

  Crawls the given URL and nested links to max_crawl_depth. Data is stored to output_path.

• llm_rag_create_faiss_index

  Creates a FAISS index from embeddings. The index will be saved to the output folder. The index will be registered as a Data Asset named asset_name if register_output is set to True.

• llm_rag_create_promptflow

  This component is used to create a RAG flow based on your mlindex data and best prompts. The flow will look into your indexed data and give answers based on your own data context. The flow also provides the capability to bulk test with any built-in or custom evaluation flows.

• llm_rag_data_import_acs

  Collects documents from Azure Cognitive Search Index, extracts their contents, saves them to a uri folder, and creates an MLIndex yaml file to represent the search index.

Documents collected can then be used in other components without having to query the ACS index again, allowing for a consiste...

• llm_rag_generate_embeddings

  Generates embeddings vectors for data chunks read from chunks_source.

chunks_source is expected to contain csv files containing two columns:

• "Chunk" - Chunk of text to be embedded
• "Metadata" - JSON object containing metadata for the chunk

If embeddings_container is supplied, input c...

• llm_rag_generate_embeddings_parallel

  Generates embeddings vectors for data chunks read from chunks_source.

chunks_source is expected to contain csv files containing two columns:

• "Chunk" - Chunk of text to be embedded
• "Metadata" - JSON object containing metadata for the chunk

If previous_embeddings is supplied, input ch...

• llm_rag_git_clone

  Clones a git repository to output_data path

• llm_rag_image_embed_index

  Embeds input images and stores it in Azure Cognitive Search index with metadata using Florence embedding resource. MLIndex is stored to output_path.

• llm_rag_qa_data_generation

  Generates a test dataset of questions and answers based on the input documents.

A chunk of text is read from each input document and sent to the specified LLM with a prompt to create a question and answer based on that text. These question, answer, and context sets are saved as either a csv or j...

• llm_rag_register_mlindex_asset

  Registers a MLIndex yaml and supporting files as an AzureML data asset

• llm_rag_register_qa_data_asset

  Registers a QA data csv or json and supporting files as an AzureML data asset

• llm_rag_update_acs_index

  Uploads embeddings into Azure Cognitive Search instance specified in acs_config. The Index will be created if it doesn't exist.

The Index will have the following fields populated:

• "id", String, key=True

• "content", String

• "contentVector", Collection(Single)

• "category", String

• "url",...

• llm_rag_update_cosmos_mongo_vcore_index

  Uploads embeddings into Azure Cosmos Mongo vCore collection/index specified in azure_cosmos_mongo_vcore_config. The collection/index will be created if it doesn't exist.

The collection/index will have the following fields populated:

• "_id", String, key=True

• "content", String

• "contentVec...

• llm_rag_update_milvus_index

  Uploads embeddings into Milvus collection/index specified in milvus_config. The collection/index will be created if it doesn't exist.

The collection/index will have the following fields populated:

• "id", String, key=True

• "content", String

• "contentVector", Collection(Single)

• "url", Str...

• llm_rag_update_pinecone_index

  Uploads embeddings into Pinecone index specified in pinecone_config. The Index will be created if it doesn't exist.

Each record in the Index will have the following metadata populated:

• "id", String

• "content", String

• "url", String

• "filepath", String

• "title", String

• "metadata_json_...

• llm_rag_validate_deployments

  Validates that completion model, embedding model, and Azure Cognitive Search resource deployments is successful and connections works. For default AOAI, it attempts to create the deployments if not valid or present. This validation is done only if customer is using Azure Open AI models or creatin...

• microsoft_azureml_rai_tabular_causal

  Add Causal to RAI Insights Dashboard Learn More

• microsoft_azureml_rai_tabular_counterfactual

  Add Counterfactuals to RAI Insights Dashboard Learn More

• microsoft_azureml_rai_tabular_erroranalysis

  Add Error Analysis to RAI Insights Dashboard Learn More

• microsoft_azureml_rai_tabular_explanation

  Add Explanation to RAI Insights Dashboard Learn More

• microsoft_azureml_rai_tabular_insight_constructor

  RAI Insights Dashboard Constructor Learn More

• microsoft_azureml_rai_tabular_insight_gather

  Gather RAI Insights Dashboard Learn More

• microsoft_azureml_rai_tabular_score_card

  Generate rai insight score card Learn More

• mlflow_model_local_validation

  Validates if a MLFLow model can be loaded on a compute and is usable for inferencing.

• mmdetection_image_objectdetection_instancesegmentation_finetune

  Component to finetune MMDetection models for image object detection and instance segmentation.

• mmdetection_image_objectdetection_instancesegmentation_model_import

  Import PyTorch / MLflow model

• mmdetection_image_objectdetection_instancesegmentation_pipeline

  Pipeline component for image object detection and instance segmentation using MMDetection models.

• mmtracking_video_multi_object_tracking_finetune

  Component to finetune MMTracking models for video multi-object tracking task.

• mmtracking_video_multi_object_tracking_model_import

  Import PyTorch / MLflow model

• mmtracking_video_multi_object_tracking_pipeline

  Pipeline component for multi-object tracking using MMTracking models.

• model_data_collector_preprocessor

  Filters the data based on the window provided.

• model_evaluation_pipeline

  Pipeline component for model evaluation for supported tasks. \ Generates predictions on a given model, followed by computing model performance metrics to score the model quality for supported tasks.

• model_monitor_action_analyzer

  Generate and output actions to the default datastore.

• model_monitor_action_detector

  Generate and output actions

• model_monitor_azmon_metric_publisher

  Azure Monitor Publisher for the computed model monitor metrics.

• model_monitor_compute_histogram

  Compute a histogram given an input data and associated histogram buckets.

• model_monitor_compute_histogram_buckets

  Compute histogram buckets given up to two datasets.

• model_monitor_create_manifest

  Creates the model monitor metric manifest.

• model_monitor_data_joiner

  Joins two data assets on the given columns for model monitor.

• model_monitor_evaluate_metrics_threshold

  Evaluate signal metrics against the threshold provided in the monitoring signal.

• model_monitor_feature_selector

  Selects features to compute signal metrics on.

• model_monitor_metric_outputter

  Output the computed model monitor metrics.

• model_monitor_output_metrics

  Output the computed model monitor metrics to the default datastore.

• model_performance_compute_metrics

  Compute model performance metrics leveraged by the model performance monitor.

• model_performance_signal_monitor

  Computes the model performance

• model_prediction

  Generate predictions on a given mlflow model for supported tasks.

• model_prediction_with_container

  Optimized Distributed inference component for LLMs.

• multimodal_classification_datapreprocessing

  Component to preprocess data for multimodal classification task

• multimodal_classification_finetune

  Component to finetune multimodal models for classification using MMEFT

• multimodal_classification_model_import

  Import PyTorch / MLflow model

• multimodal_classification_pipeline

  Pipeline component for multimodal classification models.

• my_sample

• nlp_multiclass_datapreprocessing

  Component to preprocess data for automl nlp multiclass classification task

• nlp_multilabel_datapreprocessing

  Component to preprocess data for automl nlp multilabel classification task

• nlp_ner_datapreprocessing

  Component to preprocess data for automl nlp ner task

• nlp_textclassification_multiclass

  Pipeline component for AutoML NLP Multiclass Text classification

• nlp_textclassification_multilabel

  Pipeline component for AutoML NLP Multilabel Text classification

• nlp_textclassification_ner

  Pipeline component for AutoML NLP NER

• openai_completions_finetune

  Finetune your own OAI model. Visit https://learn.microsoft.com/en-us/azure/cognitive-services/openai/ for more info.

• openai_completions_finetune_pipeline

  Finetune your own OAI model. Visit https://learn.microsoft.com/en-us/azure/cognitive-services/openai/ for more info.

• oss_chat_completion_finetune

  FTaaS component to finetune model for Chat Completion task

• oss_chat_completion_pipeline

  FTaaS Pipeline component for chat completion

• oss_distillation_generate_data

  Component to generate data from teacher model enpoint

• oss_distillation_pipeline

  Component to generate data from teacher model enpoint and finetune student model on generated dataset

• oss_distillation_validate_pipeline

  Component to validate inputs to the distillation pipeline

• oss_text_generation_data_import

  FTaaS component to copy user training data to output

• oss_text_generation_finetune

  FTaaS component to finetune model for Text Generation task

• oss_text_generation_pipeline

  FTaaS Pipeline component for text generation

• prediction_drift_signal_monitor

  Computes the prediction drift between a baseline and a target data assets.

• prompt_crafter

  This component is used to create prompts from a given dataset. From a given jinja prompt template, it will generate prompts. It can also create few-shot prompts given a few-shot dataset and the number of shots.

• question_answering_datapreprocess

  Component to preprocess data for question answering task. See docs to learn more.

• question_answering_finetune

  Component to finetune Hugging Face pretrained models for extractive question answering task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• question_answering_model_import

  Component to import PyTorch / MLFlow model. See docs to learn more.

• question_answering_pipeline

  Pipeline Component to finetune Hugging Face pretrained models for extractive question answering task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• rai_text_insights

• rai_vision_insights

• register_model

  Register a model to a workspace or a registry. The component works on compute with MSI attached.

• summarization_datapreprocess

  Component to preprocess data for summarization task. See docs to learn more.

• summarization_finetune

  Component to finetune Hugging Face pretrained models for summarization task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• summarization_model_import

  Component to import PyTorch / MLFlow model. See docs to learn more.

• summarization_pipeline

  Pipeline Component to finetune Hugging Face pretrained models for summarization task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• text_classification_datapreprocess

  Component to preprocess data for single label classification task. See docs to learn more.

• text_classification_finetune

  Component to finetune Hugging Face pretrained models for text classification task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• text_classification_model_import

  Component to import PyTorch / MLFlow model. See docs to learn more.

• text_classification_pipeline

  Pipeline component to finetune Hugging Face pretrained models for text classification task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• text_generation_datapreprocess

  Component to preprocess data for text generation task

• text_generation_finetune

  Component to finetune model for Text Generation task

• text_generation_model_import

  Import PyTorch / MLFlow model

• text_generation_pipeline

  Pipeline component for text generation

• text_generation_pipeline_singularity_basic_high

  Pipeline component for text generation

• text_generation_pipeline_singularity_basic_low

  Pipeline component for text generation

• text_generation_pipeline_singularity_basic_medium

  Pipeline component for text generation

• text_generation_pipeline_singularity_premium_high

  Pipeline component for text generation

• text_generation_pipeline_singularity_premium_low

  Pipeline component for text generation

• text_generation_pipeline_singularity_premium_medium

  Pipeline component for text generation

• text_generation_pipeline_singularity_standard_high

  Pipeline component for text generation

• text_generation_pipeline_singularity_standard_low

  Pipeline component for text generation

• text_generation_pipeline_singularity_standard_medium

  Pipeline component for text generation

• token_classification_datapreprocess

  Component to preprocess data for token classification task. See docs to learn more.

• token_classification_finetune

  Component to finetune Hugging Face pretrained models for token classification task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• token_classification_model_import

  Component to import PyTorch / MLFlow model. See docs to learn more.

• token_classification_pipeline

  Pipeline component to finetune Hugging Face pretrained models for token classification task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• token_statistics_compute_metrics

  Compute token statistics metrics.

• train_image_classification_model

  Component to finetune AutoML legacy models for image classification.

• train_instance_segmentation_model

  Component to finetune AutoML legacy models for instance segmentation.

• train_object_detection_model

  Component to finetune AutoML legacy models for object detection.

• transformers_image_classification_finetune

  Component to finetune HuggingFace transformers models for image classification.

• transformers_image_classification_model_import

  Import PyTorch / MLflow model

• transformers_image_classification_pipeline

  Pipeline component for image classification using HuggingFace transformers models.

• translation_datapreprocess

  Component to preprocess data for translation task. See docs to learn more.

• translation_finetune

  Component to finetune Hugging Face pretrained models for translation task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• translation_model_import

  Component to import PyTorch / MLFlow model. See docs to learn more.

• translation_pipeline

  Pipeline component to finetune Hugging Face pretrained models for translation task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See docs to learn more.

• validation_trigger_import

  Component for enabling validation of import pipeline.

• validation_trigger_model_evaluation

  Component for enabling validation of model evaluation pipeline.