Skip to content

Latest commit

 

History

History
48 lines (39 loc) · 2.59 KB

README.md

File metadata and controls

48 lines (39 loc) · 2.59 KB

Spatial_Intelligence_Blocks

A block-stacking task built based on robosuite.

Install

  • 首先安装robosuite，推荐git clone + pip install -e .安装，方便定制环境
  • 将si-visual.xml, siBig-visual.xml, siRed-visual.xml复制至robosuite/robosuite/models/assets/objects/目录下
  • 将table_arena_si.xml复制至robosuite/robosuite/models/assets/arenas/目录下
  • 将环境代码spatial_intelligence.py复制至robosuite/robosuite/environments/manipulation/目录下
  • 在robosuite/robosuite/__init__.py中添加代码from robosuite.environments.manipulation.spatial_intelligence import SpatialIntelligence

测试

./test.sh
  • 输出类似如下内容
result: True, message: Cube is placed at position [1 0 0].
result: True, message: Cube is placed at position [2 0 0].
result: True, message: Cube is placed at position [3 0 0].
result: True, message: Cube is placed at position [4 0 0].
result: True, message: Cube is placed at position [5 0 0].
result: True, message: Cube is placed at position [6 0 0].
result: True, message: Cursor is moved to position [5 0 0].
...
  • 生成task_view目录，且目录中包含随机生成的积木cube_xyz_idx的三个视角图
  • 生成video目录，且目录中包含挪动光标搭建随机生成的积木cube_xyz_idx的视频

则表示环境可用

generated connected cubes

已经用CUDA_VISIBLE_DEVICES=0 python gen_connected_cube_set.py事先生成了目标积木，搭建对应积木的action序列，以及得出action序列的思考过程，存储在connected_cube_set_1250_traintest-w_thinks.pkl中。访问connected_cube_set_1250_traintest-w_thinks.pkl的方式如下：

dataset = 1250
traintest = "train"
generated_connected_cube_set = pkl.load(open(os.path.join('$data_dir$', f"connected_cube_set_{dataset}_traintest-w_thinks.pkl")), "rb")[traintest]

for eps in generated_connected_cube_set.keys():
    eps_data = generated_connected_cube_set[eps]
    eps_task = eps_data['task'] # 格式是一个Nx * Ny * Nz的0/1数组，1构成目标积木的外观
    eps_solution = eps_data['solution'] # 格式是一个action list，每个action对应一个step采取的action
    eps_think = eps_data['think'] # 格式是一个str list，每个str对应一个step的reasoning process

此外也提供了测试caseconnected_cube_set_50_traintest-w_thinks.pkl供复现，基于sft数据集的每个task对应目标积木的三视图，和搭建积木的过程可视化在./temp/目录下。用CUDA_VISIBLE_DEVICES=0 run_gened_connected_cube_set.py进行复现