AWS Deep Learning Containers (DLCs) are a set of Docker images for training and serving models in TensorFlow, TensorFlow 2, PyTorch, and MXNet.

A universal scalable machine learning model deployment solution

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.

Benchmark suite for LLMs from Fireworks.ai

A framework for few-shot evaluation of language models.

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.

Model Serving on PyTorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Toolkit for running PyTorch training scripts on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at https://github.com/aws/deep-learning-containers.

An open-source C++ library developed and used at Facebook.

A GPU-powered real-time analytics storage and query engine.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Multi Model Server is a tool for serving neural net models for inference

Blazing fast, structured, leveled logging in Go.