Embodied-AI

This repository mainly organizes resources related to embodied intelligence, including data, models, hardware, and software infrastructure.

Data

3D Grounding

SCENEVERSE: Scaling 3D Vision-Language Learning for Grounded Scene Understanding [arxiv] [github] [2024]
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language [arxiv] [website] [2019]

LLM-Robot

AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents [paper] [website] [2024]
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks [arxiv] [github] [2021]
Open X-Embodiment: Robotic Learning Datasets and RT-X Models [website] [github]

Model

3D Grounding

LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent [arxiv] [github] [2023]
Multi-View Transformer for 3D Visual Grounding [paper] [github] [2022]

LLM-Robot

Generative Expressive Robot Behaviors Using Large Language Models [arxiv] [website] [2024]
OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics [arxiv] [github] [2024]
MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D World [arxiv] [website] [2024]
VISION-LANGUAGE FOUNDATION MODELS AS EFFEC-TIVE ROBOT IMITATORS [arxiv] [github] [2023]

Task Planning

Embodied Task Planning with Large Language Models [arxiv] [website] [2023]

Hardware

Soc

Software

AI Infrastructure

Large Scale Training Framework(based Pytorch)

DeepSpeed

Edge Inference Engine

TVM